|
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606 |
- title: What happens when...
- url: https://github.com/alex/what-happens-when/blob/master/README.rst
- hash_url: 46f440a287ca07d71b0fd3e7915a9a0c
-
- <article class="markdown-body entry-content" itemprop="text"><a name="user-content-what-happens-when"/>
- <h2><a id="user-content-what-happens-when" class="anchor" aria-hidden="true" href="#what-happens-when"><svg class="octicon octicon-link" viewbox="0 0 16 16" version="1.1" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"/></svg></a>What happens when...</h2>
- <p>This repository is an attempt to answer the age old interview question "What
- happens when you type google.com into your browser's address box and press
- enter?"</p>
- <p>Except instead of the usual story, we're going to try to answer this question
- in as much detail as possible. No skipping out on anything.</p>
- <p>This is a collaborative process, so dig in and try to help out! There are tons
- of details missing, just waiting for you to add them! So send us a pull
- request, please!</p>
- <p>This is all licensed under the terms of the <a href="https://creativecommons.org/publicdomain/zero/1.0/" rel="nofollow">Creative Commons Zero</a> license.</p>
- <p>Read this in <a href="https://github.com/skyline75489/what-happens-when-zh_CN">简体中文</a> (simplified Chinese) and <a href="https://github.com/SantonyChoi/what-happens-when-KR">한국어</a> (Korean). NOTE: these
- have not been reviewed by the alex/what-happens-when maintainers.</p>
- <a name="user-content-table-of-contents"/>
- <h2><a id="user-content-table-of-contents" class="anchor" aria-hidden="true" href="#table-of-contents"><svg class="octicon octicon-link" viewbox="0 0 16 16" version="1.1" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"/></svg></a>Table of Contents</h2>
-
- <a name="user-content-the-g-key-is-pressed"/>
- <h3><a id="user-content-the-g-key-is-pressed" class="anchor" aria-hidden="true" href="#the-g-key-is-pressed"><svg class="octicon octicon-link" viewbox="0 0 16 16" version="1.1" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"/></svg></a>The "g" key is pressed</h3>
- <p>The following sections explain the physical keyboard actions
- and the OS interrupts. When you press the key "g" the browser receives the
- event and the auto-complete functions kick in.
- Depending on your browser's algorithm and if you are in
- private/incognito mode or not various suggestions will be presented
- to you in the dropbox below the URL bar. Most of these algorithms sort
- and prioritize results based on search history, bookmarks, cookies, and
- popular searches from the internet as a whole. As you are typing
- "google.com" many blocks of code run and the suggestions will be refined
- with each key press. It may even suggest "google.com" before you finish typing
- it.</p>
- <a name="user-content-the-enter-key-bottoms-out"/>
- <h3><a id="user-content-the-enter-key-bottoms-out" class="anchor" aria-hidden="true" href="#the-enter-key-bottoms-out"><svg class="octicon octicon-link" viewbox="0 0 16 16" version="1.1" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"/></svg></a>The "enter" key bottoms out</h3>
- <p>To pick a zero point, let's choose the Enter key on the keyboard hitting the
- bottom of its range. At this point, an electrical circuit specific to the enter
- key is closed (either directly or capacitively). This allows a small amount of
- current to flow into the logic circuitry of the keyboard, which scans the state
- of each key switch, debounces the electrical noise of the rapid intermittent
- closure of the switch, and converts it to a keycode integer, in this case 13.
- The keyboard controller then encodes the keycode for transport to the computer.
- This is now almost universally over a Universal Serial Bus (USB) or Bluetooth
- connection, but historically has been over PS/2 or ADB connections.</p>
- <p><em>In the case of the USB keyboard:</em></p>
- <ul>
- <li>The USB circuitry of the keyboard is powered by the 5V supply provided over
- pin 1 from the computer's USB host controller.</li>
- <li>The keycode generated is stored by internal keyboard circuitry memory in a
- register called "endpoint".</li>
- <li>The host USB controller polls that "endpoint" every ~10ms (minimum value
- declared by the keyboard), so it gets the keycode value stored on it.</li>
- <li>This value goes to the USB SIE (Serial Interface Engine) to be converted in
- one or more USB packets that follow the low level USB protocol.</li>
- <li>Those packets are sent by a differential electrical signal over D+ and D-
- pins (the middle 2) at a maximum speed of 1.5 Mb/s, as an HID
- (Human Interface Device) device is always declared to be a "low speed device"
- (USB 2.0 compliance).</li>
- <li>This serial signal is then decoded at the computer's host USB controller, and
- interpreted by the computer's Human Interface Device (HID) universal keyboard
- device driver. The value of the key is then passed into the operating
- system's hardware abstraction layer.</li>
- </ul>
- <p><em>In the case of Virtual Keyboard (as in touch screen devices):</em></p>
- <ul>
- <li>When the user puts their finger on a modern capacitive touch screen, a
- tiny amount of current gets transferred to the finger. This completes the
- circuit through the electrostatic field of the conductive layer and
- creates a voltage drop at that point on the screen. The
- <code>screen controller</code> then raises an interrupt reporting the coordinate of
- the key press.</li>
- <li>Then the mobile OS notifies the current focused application of a press event
- in one of its GUI elements (which now is the virtual keyboard application
- buttons).</li>
- <li>The virtual keyboard can now raise a software interrupt for sending a
- 'key pressed' message back to the OS.</li>
- <li>This interrupt notifies the current focused application of a 'key pressed'
- event.</li>
- </ul>
- <a name="user-content-interrupt-fires-not-for-usb-keyboards"/>
- <h3><a id="user-content-interrupt-fires-not-for-usb-keyboards" class="anchor" aria-hidden="true" href="#interrupt-fires-not-for-usb-keyboards"><svg class="octicon octicon-link" viewbox="0 0 16 16" version="1.1" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"/></svg></a>Interrupt fires [NOT for USB keyboards]</h3>
- <p>The keyboard sends signals on its interrupt request line (IRQ), which is mapped
- to an <code>interrupt vector</code> (integer) by the interrupt controller. The CPU uses
- the <code>Interrupt Descriptor Table</code> (IDT) to map the interrupt vectors to
- functions (<code>interrupt handlers</code>) which are supplied by the kernel. When an
- interrupt arrives, the CPU indexes the IDT with the interrupt vector and runs
- the appropriate handler. Thus, the kernel is entered.</p>
- <a name="user-content-on-windows-a-wm-keydown-message-is-sent-to-the-app"/>
- <h3><a id="user-content-on-windows-a-wm_keydown-message-is-sent-to-the-app" class="anchor" aria-hidden="true" href="#on-windows-a-wm_keydown-message-is-sent-to-the-app"><svg class="octicon octicon-link" viewbox="0 0 16 16" version="1.1" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"/></svg></a>(On Windows) A <code>WM_KEYDOWN</code> message is sent to the app</h3>
- <p>The HID transport passes the key down event to the <code>KBDHID.sys</code> driver which
- converts the HID usage into a scancode. In this case the scan code is
- <code>VK_RETURN</code> (<code>0x0D</code>). The <code>KBDHID.sys</code> driver interfaces with the
- <code>KBDCLASS.sys</code> (keyboard class driver). This driver is responsible for
- handling all keyboard and keypad input in a secure manner. It then calls into
- <code>Win32K.sys</code> (after potentially passing the message through 3rd party
- keyboard filters that are installed). This all happens in kernel mode.</p>
- <p><code>Win32K.sys</code> figures out what window is the active window through the
- <code>GetForegroundWindow()</code> API. This API provides the window handle of the
- browser's address box. The main Windows "message pump" then calls
- <code>SendMessage(hWnd, WM_KEYDOWN, VK_RETURN, lParam)</code>. <code>lParam</code> is a bitmask
- that indicates further information about the keypress: repeat count (0 in this
- case), the actual scan code (can be OEM dependent, but generally wouldn't be
- for <code>VK_RETURN</code>), whether extended keys (e.g. alt, shift, ctrl) were also
- pressed (they weren't), and some other state.</p>
- <p>The Windows <code>SendMessage</code> API is a straightforward function that
- adds the message to a queue for the particular window handle (<code>hWnd</code>).
- Later, the main message processing function (called a <code>WindowProc</code>) assigned
- to the <code>hWnd</code> is called in order to process each message in the queue.</p>
- <p>The window (<code>hWnd</code>) that is active is actually an edit control and the
- <code>WindowProc</code> in this case has a message handler for <code>WM_KEYDOWN</code> messages.
- This code looks within the 3rd parameter that was passed to <code>SendMessage</code>
- (<code>wParam</code>) and, because it is <code>VK_RETURN</code> knows the user has hit the ENTER
- key.</p>
- <a name="user-content-on-os-x-a-keydown-nsevent-is-sent-to-the-app"/>
- <h3><a id="user-content-on-os-x-a-keydown-nsevent-is-sent-to-the-app" class="anchor" aria-hidden="true" href="#on-os-x-a-keydown-nsevent-is-sent-to-the-app"><svg class="octicon octicon-link" viewbox="0 0 16 16" version="1.1" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"/></svg></a>(On OS X) A <code>KeyDown</code> NSEvent is sent to the app</h3>
- <p>The interrupt signal triggers an interrupt event in the I/O Kit kext keyboard
- driver. The driver translates the signal into a key code which is passed to the
- OS X <code>WindowServer</code> process. Resultantly, the <code>WindowServer</code> dispatches an
- event to any appropriate (e.g. active or listening) applications through their
- Mach port where it is placed into an event queue. Events can then be read from
- this queue by threads with sufficient privileges calling the
- <code>mach_ipc_dispatch</code> function. This most commonly occurs through, and is
- handled by, an <code>NSApplication</code> main event loop, via an <code>NSEvent</code> of
- <code>NSEventType</code> <code>KeyDown</code>.</p>
- <a name="user-content-on-gnu-linux-the-xorg-server-listens-for-keycodes"/>
- <h3><a id="user-content-on-gnulinux-the-xorg-server-listens-for-keycodes" class="anchor" aria-hidden="true" href="#on-gnulinux-the-xorg-server-listens-for-keycodes"><svg class="octicon octicon-link" viewbox="0 0 16 16" version="1.1" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"/></svg></a>(On GNU/Linux) the Xorg server listens for keycodes</h3>
- <p>When a graphical <code>X server</code> is used, <code>X</code> will use the generic event
- driver <code>evdev</code> to acquire the keypress. A re-mapping of keycodes to scancodes
- is made with <code>X server</code> specific keymaps and rules.
- When the scancode mapping of the key pressed is complete, the <code>X server</code>
- sends the character to the <code>window manager</code> (DWM, metacity, i3, etc), so the
- <code>window manager</code> in turn sends the character to the focused window.
- The graphical API of the window that receives the character prints the
- appropriate font symbol in the appropriate focused field.</p>
- <a name="user-content-parse-url"/>
- <h3><a id="user-content-parse-url" class="anchor" aria-hidden="true" href="#parse-url"><svg class="octicon octicon-link" viewbox="0 0 16 16" version="1.1" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"/></svg></a>Parse URL</h3>
-
- <a name="user-content-is-it-a-url-or-a-search-term"/>
- <h3><a id="user-content-is-it-a-url-or-a-search-term" class="anchor" aria-hidden="true" href="#is-it-a-url-or-a-search-term"><svg class="octicon octicon-link" viewbox="0 0 16 16" version="1.1" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"/></svg></a>Is it a URL or a search term?</h3>
- <p>When no protocol or valid domain name is given the browser proceeds to feed
- the text given in the address box to the browser's default web search engine.
- In many cases the URL has a special piece of text appended to it to tell the
- search engine that it came from a particular browser's URL bar.</p>
- <a name="user-content-convert-non-ascii-unicode-characters-in-hostname"/>
- <h3><a id="user-content-convert-non-ascii-unicode-characters-in-hostname" class="anchor" aria-hidden="true" href="#convert-non-ascii-unicode-characters-in-hostname"><svg class="octicon octicon-link" viewbox="0 0 16 16" version="1.1" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"/></svg></a>Convert non-ASCII Unicode characters in hostname</h3>
- <ul>
- <li>The browser checks the hostname for characters that are not in <code>a-z</code>,
- <code>A-Z</code>, <code>0-9</code>, <code>-</code>, or <code>.</code>.</li>
- <li>Since the hostname is <code>google.com</code> there won't be any, but if there were
- the browser would apply <a href="https://en.wikipedia.org/wiki/Punycode" rel="nofollow">Punycode</a> encoding to the hostname portion of the
- URL.</li>
- </ul>
- <a name="user-content-check-hsts-list"/>
- <h3><a id="user-content-check-hsts-list" class="anchor" aria-hidden="true" href="#check-hsts-list"><svg class="octicon octicon-link" viewbox="0 0 16 16" version="1.1" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"/></svg></a>Check HSTS list</h3>
- <ul>
- <li>The browser checks its "preloaded HSTS (HTTP Strict Transport Security)"
- list. This is a list of websites that have requested to be contacted via
- HTTPS only.</li>
- <li>If the website is in the list, the browser sends its request via HTTPS
- instead of HTTP. Otherwise, the initial request is sent via HTTP.
- (Note that a website can still use the HSTS policy <em>without</em> being in the
- HSTS list. The first HTTP request to the website by a user will receive a
- response requesting that the user only send HTTPS requests. However, this
- single HTTP request could potentially leave the user vulnerable to a
- <a href="http://en.wikipedia.org/wiki/SSL_stripping" rel="nofollow">downgrade attack</a>, which is why the HSTS list is included in modern web
- browsers.)</li>
- </ul>
- <a name="user-content-dns-lookup"/>
- <h3><a id="user-content-dns-lookup" class="anchor" aria-hidden="true" href="#dns-lookup"><svg class="octicon octicon-link" viewbox="0 0 16 16" version="1.1" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"/></svg></a>DNS lookup</h3>
- <ul>
- <li>Browser checks if the domain is in its cache. (to see the DNS Cache in
- Chrome, go to chrome://net-internals/#dns).</li>
- <li>If not found, the browser calls <code>gethostbyname</code> library function (varies by
- OS) to do the lookup.</li>
- <li><code>gethostbyname</code> checks if the hostname can be resolved by reference in the
- local <code>hosts</code> file (whose location <a href="https://en.wikipedia.org/wiki/Hosts_%28file%29#Location_in_the_file_system" rel="nofollow">varies by OS</a>) before trying to
- resolve the hostname through DNS.</li>
- <li>If <code>gethostbyname</code> does not have it cached nor can find it in the <code>hosts</code>
- file then it makes a request to the DNS server configured in the network
- stack. This is typically the local router or the ISP's caching DNS server.</li>
- <li>If the DNS server is on the same subnet the network library follows the
- <code>ARP process</code> below for the DNS server.</li>
- <li>If the DNS server is on a different subnet, the network library follows
- the <code>ARP process</code> below for the default gateway IP.</li>
- </ul>
- <a name="user-content-arp-process"/>
- <h3><a id="user-content-arp-process" class="anchor" aria-hidden="true" href="#arp-process"><svg class="octicon octicon-link" viewbox="0 0 16 16" version="1.1" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"/></svg></a>ARP process</h3>
- <p>In order to send an ARP (Address Resolution Protocol) broadcast the network
- stack library needs the target IP address to look up. It also needs to know the
- MAC address of the interface it will use to send out the ARP broadcast.</p>
- <p>The ARP cache is first checked for an ARP entry for our target IP. If it is in
- the cache, the library function returns the result: Target IP = MAC.</p>
- <p>If the entry is not in the ARP cache:</p>
- <ul>
- <li>The route table is looked up, to see if the Target IP address is on any of
- the subnets on the local route table. If it is, the library uses the
- interface associated with that subnet. If it is not, the library uses the
- interface that has the subnet of our default gateway.</li>
- <li>The MAC address of the selected network interface is looked up.</li>
- <li>The network library sends a Layer 2 (data link layer of the <a href="https://en.wikipedia.org/wiki/OSI_model" rel="nofollow">OSI model</a>)
- ARP request:</li>
- </ul>
- <p><code>ARP Request</code>:</p>
- <pre>Sender MAC: interface:mac:address:here
- Sender IP: interface.ip.goes.here
- Target MAC: FF:FF:FF:FF:FF:FF (Broadcast)
- Target IP: target.ip.goes.here
- </pre>
- <p>Depending on what type of hardware is between the computer and the router:</p>
- <p>Directly connected:</p>
- <ul>
- <li>If the computer is directly connected to the router the router responds
- with an <code>ARP Reply</code> (see below)</li>
- </ul>
- <p>Hub:</p>
- <ul>
- <li>If the computer is connected to a hub, the hub will broadcast the ARP
- request out all other ports. If the router is connected on the same "wire",
- it will respond with an <code>ARP Reply</code> (see below).</li>
- </ul>
- <p>Switch:</p>
- <ul>
- <li>If the computer is connected to a switch, the switch will check its local
- CAM/MAC table to see which port has the MAC address we are looking for. If
- the switch has no entry for the MAC address it will rebroadcast the ARP
- request to all other ports.</li>
- <li>If the switch has an entry in the MAC/CAM table it will send the ARP request
- to the port that has the MAC address we are looking for.</li>
- <li>If the router is on the same "wire", it will respond with an <code>ARP Reply</code>
- (see below)</li>
- </ul>
- <p><code>ARP Reply</code>:</p>
- <pre>Sender MAC: target:mac:address:here
- Sender IP: target.ip.goes.here
- Target MAC: interface:mac:address:here
- Target IP: interface.ip.goes.here
- </pre>
- <p>Now that the network library has the IP address of either our DNS server or
- the default gateway it can resume its DNS process:</p>
- <ul>
- <li>Port 53 is opened to send a UDP request to DNS server (if the response size
- is too large, TCP will be used instead).</li>
- <li>If the local/ISP DNS server does not have it, then a recursive search is
- requested and that flows up the list of DNS servers until the SOA is reached,
- and if found an answer is returned.</li>
- </ul>
- <a name="user-content-opening-of-a-socket"/>
- <h3><a id="user-content-opening-of-a-socket" class="anchor" aria-hidden="true" href="#opening-of-a-socket"><svg class="octicon octicon-link" viewbox="0 0 16 16" version="1.1" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"/></svg></a>Opening of a socket</h3>
- <p>Once the browser receives the IP address of the destination server, it takes
- that and the given port number from the URL (the HTTP protocol defaults to port
- 80, and HTTPS to port 443), and makes a call to the system library function
- named <code>socket</code> and requests a TCP socket stream - <code>AF_INET/AF_INET6</code> and
- <code>SOCK_STREAM</code>.</p>
- <ul>
- <li>This request is first passed to the Transport Layer where a TCP segment is
- crafted. The destination port is added to the header, and a source port is
- chosen from within the kernel's dynamic port range (ip_local_port_range in
- Linux).</li>
- <li>This segment is sent to the Network Layer, which wraps an additional IP
- header. The IP address of the destination server as well as that of the
- current machine is inserted to form a packet.</li>
- <li>The packet next arrives at the Link Layer. A frame header is added that
- includes the MAC address of the machine's NIC as well as the MAC address of
- the gateway (local router). As before, if the kernel does not know the MAC
- address of the gateway, it must broadcast an ARP query to find it.</li>
- </ul>
- <p>At this point the packet is ready to be transmitted through either:</p>
-
- <p>For most home or small business Internet connections the packet will pass from
- your computer, possibly through a local network, and then through a modem
- (MOdulator/DEModulator) which converts digital 1's and 0's into an analog
- signal suitable for transmission over telephone, cable, or wireless telephony
- connections. On the other end of the connection is another modem which converts
- the analog signal back into digital data to be processed by the next <a href="https://en.wikipedia.org/wiki/Computer_network#Network_nodes" rel="nofollow">network
- node</a> where the from and to addresses would be analyzed further.</p>
- <p>Most larger businesses and some newer residential connections will have fiber
- or direct Ethernet connections in which case the data remains digital and
- is passed directly to the next <a href="https://en.wikipedia.org/wiki/Computer_network#Network_nodes" rel="nofollow">network node</a> for processing.</p>
- <p>Eventually, the packet will reach the router managing the local subnet. From
- there, it will continue to travel to the autonomous system's (AS) border
- routers, other ASes, and finally to the destination server. Each router along
- the way extracts the destination address from the IP header and routes it to
- the appropriate next hop. The time to live (TTL) field in the IP header is
- decremented by one for each router that passes. The packet will be dropped if
- the TTL field reaches zero or if the current router has no space in its queue
- (perhaps due to network congestion).</p>
- <p>This send and receive happens multiple times following the TCP connection flow:</p>
- <ul>
- <li>Client chooses an initial sequence number (ISN) and sends the packet to the
- server with the SYN bit set to indicate it is setting the ISN</li>
- <li><dl>
- <dt>Server receives SYN and if it's in an agreeable mood:</dt>
- <dd><ul>
- <li>Server chooses its own initial sequence number</li>
- <li>Server sets SYN to indicate it is choosing its ISN</li>
- <li>Server copies the (client ISN +1) to its ACK field and adds the ACK flag
- to indicate it is acknowledging receipt of the first packet</li>
- </ul>
- </dd>
- </dl>
- </li>
- <li><dl>
- <dt>Client acknowledges the connection by sending a packet:</dt>
- <dd><ul>
- <li>Increases its own sequence number</li>
- <li>Increases the receiver acknowledgment number</li>
- <li>Sets ACK field</li>
- </ul>
- </dd>
- </dl>
- </li>
- <li><dl>
- <dt>Data is transferred as follows:</dt>
- <dd><ul>
- <li>As one side sends N data bytes, it increases its SEQ by that number</li>
- <li>When the other side acknowledges receipt of that packet (or a string of
- packets), it sends an ACK packet with the ACK value equal to the last
- received sequence from the other</li>
- </ul>
- </dd>
- </dl>
- </li>
- <li><dl>
- <dt>To close the connection:</dt>
- <dd><ul>
- <li>The closer sends a FIN packet</li>
- <li>The other sides ACKs the FIN packet and sends its own FIN</li>
- <li>The closer acknowledges the other side's FIN with an ACK</li>
- </ul>
- </dd>
- </dl>
- </li>
- </ul>
- <a name="user-content-tls-handshake"/>
- <h3><a id="user-content-tls-handshake" class="anchor" aria-hidden="true" href="#tls-handshake"><svg class="octicon octicon-link" viewbox="0 0 16 16" version="1.1" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"/></svg></a>TLS handshake</h3>
- <ul>
- <li>The client computer sends a <code>ClientHello</code> message to the server with its
- Transport Layer Security (TLS) version, list of cipher algorithms and
- compression methods available.</li>
- <li>The server replies with a <code>ServerHello</code> message to the client with the
- TLS version, selected cipher, selected compression methods and the server's
- public certificate signed by a CA (Certificate Authority). The certificate
- contains a public key that will be used by the client to encrypt the rest of
- the handshake until a symmetric key can be agreed upon.</li>
- <li>The client verifies the server digital certificate against its list of
- trusted CAs. If trust can be established based on the CA, the client
- generates a string of pseudo-random bytes and encrypts this with the server's
- public key. These random bytes can be used to determine the symmetric key.</li>
- <li>The server decrypts the random bytes using its private key and uses these
- bytes to generate its own copy of the symmetric master key.</li>
- <li>The client sends a <code>Finished</code> message to the server, encrypting a hash of
- the transmission up to this point with the symmetric key.</li>
- <li>The server generates its own hash, and then decrypts the client-sent hash
- to verify that it matches. If it does, it sends its own <code>Finished</code> message
- to the client, also encrypted with the symmetric key.</li>
- <li>From now on the TLS session transmits the application (HTTP) data encrypted
- with the agreed symmetric key.</li>
- </ul>
- <a name="user-content-http-protocol"/>
- <h3><a id="user-content-http-protocol" class="anchor" aria-hidden="true" href="#http-protocol"><svg class="octicon octicon-link" viewbox="0 0 16 16" version="1.1" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"/></svg></a>HTTP protocol</h3>
- <p>If the web browser used was written by Google, instead of sending an HTTP
- request to retrieve the page, it will send a request to try and negotiate with
- the server an "upgrade" from HTTP to the SPDY protocol.</p>
- <p>If the client is using the HTTP protocol and does not support SPDY, it sends a
- request to the server of the form:</p>
- <pre>GET / HTTP/1.1
- Host: google.com
- Connection: close
- [other headers]
- </pre>
- <p>where <code>[other headers]</code> refers to a series of colon-separated key-value pairs
- formatted as per the HTTP specification and separated by single new lines.
- (This assumes the web browser being used doesn't have any bugs violating the
- HTTP spec. This also assumes that the web browser is using <code>HTTP/1.1</code>,
- otherwise it may not include the <code>Host</code> header in the request and the version
- specified in the <code>GET</code> request will either be <code>HTTP/1.0</code> or <code>HTTP/0.9</code>.)</p>
- <p>HTTP/1.1 defines the "close" connection option for the sender to signal that
- the connection will be closed after completion of the response. For example,</p>
- <blockquote>
- Connection: close</blockquote>
- <p>HTTP/1.1 applications that do not support persistent connections MUST include
- the "close" connection option in every message.</p>
- <p>After sending the request and headers, the web browser sends a single blank
- newline to the server indicating that the content of the request is done.</p>
- <p>The server responds with a response code denoting the status of the request and
- responds with a response of the form:</p>
- <pre>200 OK
- [response headers]
- </pre>
- <p>Followed by a single newline, and then sends a payload of the HTML content of
- <code>www.google.com</code>. The server may then either close the connection, or if
- headers sent by the client requested it, keep the connection open to be reused
- for further requests.</p>
- <p>If the HTTP headers sent by the web browser included sufficient information for
- the web server to determine if the version of the file cached by the web
- browser has been unmodified since the last retrieval (ie. if the web browser
- included an <code>ETag</code> header), it may instead respond with a request of
- the form:</p>
- <pre>304 Not Modified
- [response headers]
- </pre>
- <p>and no payload, and the web browser instead retrieves the HTML from its cache.</p>
- <p>After parsing the HTML, the web browser (and server) repeats this process
- for every resource (image, CSS, favicon.ico, etc) referenced by the HTML page,
- except instead of <code>GET / HTTP/1.1</code> the request will be
- <code>GET /$(URL relative to www.google.com) HTTP/1.1</code>.</p>
- <p>If the HTML referenced a resource on a different domain than
- <code>www.google.com</code>, the web browser goes back to the steps involved in
- resolving the other domain, and follows all steps up to this point for that
- domain. The <code>Host</code> header in the request will be set to the appropriate
- server name instead of <code>google.com</code>.</p>
- <a name="user-content-http-server-request-handle"/>
- <h3><a id="user-content-http-server-request-handle" class="anchor" aria-hidden="true" href="#http-server-request-handle"><svg class="octicon octicon-link" viewbox="0 0 16 16" version="1.1" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"/></svg></a>HTTP Server Request Handle</h3>
- <p>The HTTPD (HTTP Daemon) server is the one handling the requests/responses on
- the server side. The most common HTTPD servers are Apache or nginx for Linux
- and IIS for Windows.</p>
- <ul>
- <li>The HTTPD (HTTP Daemon) receives the request.</li>
- <li><dl>
- <dt>The server breaks down the request to the following parameters:</dt>
- <dd><ul>
- <li>HTTP Request Method (either <code>GET</code>, <code>HEAD</code>, <code>POST</code>, <code>PUT</code>,
- <code>DELETE</code>, <code>CONNECT</code>, <code>OPTIONS</code>, or <code>TRACE</code>). In the case of a URL
- entered directly into the address bar, this will be <code>GET</code>.</li>
- <li>Domain, in this case - google.com.</li>
- <li>Requested path/page, in this case - / (as no specific path/page was
- requested, / is the default path).</li>
- </ul>
- </dd>
- </dl>
- </li>
- <li>The server verifies that there is a Virtual Host configured on the server
- that corresponds with google.com.</li>
- <li>The server verifies that google.com can accept GET requests.</li>
- <li>The server verifies that the client is allowed to use this method
- (by IP, authentication, etc.).</li>
- <li>If the server has a rewrite module installed (like mod_rewrite for Apache or
- URL Rewrite for IIS), it tries to match the request against one of the
- configured rules. If a matching rule is found, the server uses that rule to
- rewrite the request.</li>
- <li>The server goes to pull the content that corresponds with the request,
- in our case it will fall back to the index file, as "/" is the main file
- (some cases can override this, but this is the most common method).</li>
- <li>The server parses the file according to the handler. If Google
- is running on PHP, the server uses PHP to interpret the index file, and
- streams the output to the client.</li>
- </ul>
- <a name="user-content-behind-the-scenes-of-the-browser"/>
- <h3><a id="user-content-behind-the-scenes-of-the-browser" class="anchor" aria-hidden="true" href="#behind-the-scenes-of-the-browser"><svg class="octicon octicon-link" viewbox="0 0 16 16" version="1.1" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"/></svg></a>Behind the scenes of the Browser</h3>
- <p>Once the server supplies the resources (HTML, CSS, JS, images, etc.)
- to the browser it undergoes the below process:</p>
- <ul>
- <li>Parsing - HTML, CSS, JS</li>
- <li>Rendering - Construct DOM Tree → Render Tree → Layout of Render Tree →
- Painting the render tree</li>
- </ul>
- <a name="user-content-browser"/>
- <h3><a id="user-content-browser" class="anchor" aria-hidden="true" href="#browser"><svg class="octicon octicon-link" viewbox="0 0 16 16" version="1.1" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"/></svg></a>Browser</h3>
- <p>The browser's functionality is to present the web resource you choose, by
- requesting it from the server and displaying it in the browser window.
- The resource is usually an HTML document, but may also be a PDF,
- image, or some other type of content. The location of the resource is
- specified by the user using a URI (Uniform Resource Identifier).</p>
- <p>The way the browser interprets and displays HTML files is specified
- in the HTML and CSS specifications. These specifications are maintained
- by the W3C (World Wide Web Consortium) organization, which is the
- standards organization for the web.</p>
- <p>Browser user interfaces have a lot in common with each other. Among the
- common user interface elements are:</p>
- <ul>
- <li>An address bar for inserting a URI</li>
- <li>Back and forward buttons</li>
- <li>Bookmarking options</li>
- <li>Refresh and stop buttons for refreshing or stopping the loading of
- current documents</li>
- <li>Home button that takes you to your home page</li>
- </ul>
- <p><strong>Browser High Level Structure</strong></p>
- <p>The components of the browsers are:</p>
- <ul>
- <li><strong>User interface:</strong> The user interface includes the address bar,
- back/forward button, bookmarking menu, etc. Every part of the browser
- display except the window where you see the requested page.</li>
- <li><strong>Browser engine:</strong> The browser engine marshals actions between the UI
- and the rendering engine.</li>
- <li><strong>Rendering engine:</strong> The rendering engine is responsible for displaying
- requested content. For example if the requested content is HTML, the
- rendering engine parses HTML and CSS, and displays the parsed content on
- the screen.</li>
- <li><strong>Networking:</strong> The networking handles network calls such as HTTP requests,
- using different implementations for different platforms behind a
- platform-independent interface.</li>
- <li><strong>UI backend:</strong> The UI backend is used for drawing basic widgets like combo
- boxes and windows. This backend exposes a generic interface that is not
- platform specific.
- Underneath it uses operating system user interface methods.</li>
- <li><strong>JavaScript engine:</strong> The JavaScript engine is used to parse and
- execute JavaScript code.</li>
- <li><strong>Data storage:</strong> The data storage is a persistence layer. The browser may
- need to save all sorts of data locally, such as cookies. Browsers also
- support storage mechanisms such as localStorage, IndexedDB, WebSQL and
- FileSystem.</li>
- </ul>
- <a name="user-content-html-parsing"/>
- <h3><a id="user-content-html-parsing" class="anchor" aria-hidden="true" href="#html-parsing"><svg class="octicon octicon-link" viewbox="0 0 16 16" version="1.1" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"/></svg></a>HTML parsing</h3>
- <p>The rendering engine starts getting the contents of the requested
- document from the networking layer. This will usually be done in 8kB chunks.</p>
- <p>The primary job of HTML parser to parse the HTML markup into a parse tree.</p>
- <p>The output tree (the "parse tree") is a tree of DOM element and attribute
- nodes. DOM is short for Document Object Model. It is the object presentation
- of the HTML document and the interface of HTML elements to the outside world
- like JavaScript. The root of the tree is the "Document" object. Prior of
- any manipulation via scripting, the DOM has an almost one-to-one relation to
- the markup.</p>
- <p><strong>The parsing algorithm</strong></p>
- <p>HTML cannot be parsed using the regular top-down or bottom-up parsers.</p>
- <p>The reasons are:</p>
- <ul>
- <li>The forgiving nature of the language.</li>
- <li>The fact that browsers have traditional error tolerance to support well
- known cases of invalid HTML.</li>
- <li>The parsing process is reentrant. For other languages, the source doesn't
- change during parsing, but in HTML, dynamic code (such as script elements
- containing document.write() calls) can add extra tokens, so the parsing
- process actually modifies the input.</li>
- </ul>
- <p>Unable to use the regular parsing techniques, the browser utilizes a custom
- parser for parsing HTML. The parsing algorithm is described in
- detail by the HTML5 specification.</p>
- <p>The algorithm consists of two stages: tokenization and tree construction.</p>
- <p><strong>Actions when the parsing is finished</strong></p>
- <p>The browser begins fetching external resources linked to the page (CSS, images,
- JavaScript files, etc.).</p>
- <p>At this stage the browser marks the document as interactive and starts
- parsing scripts that are in "deferred" mode: those that should be
- executed after the document is parsed. The document state is
- set to "complete" and a "load" event is fired.</p>
- <p>Note there is never an "Invalid Syntax" error on an HTML page. Browsers fix
- any invalid content and go on.</p>
- <a name="user-content-css-interpretation"/>
- <h3><a id="user-content-css-interpretation" class="anchor" aria-hidden="true" href="#css-interpretation"><svg class="octicon octicon-link" viewbox="0 0 16 16" version="1.1" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"/></svg></a>CSS interpretation</h3>
- <ul>
- <li>Parse CSS files, <code><style></code> tag contents, and <code>style</code> attribute
- values using <a href="http://www.w3.org/TR/CSS2/grammar.html" rel="nofollow">"CSS lexical and syntax grammar"</a></li>
- <li>Each CSS file is parsed into a <code>StyleSheet object</code>, where each object
- contains CSS rules with selectors and objects corresponding CSS grammar.</li>
- <li>A CSS parser can be top-down or bottom-up when a specific parser generator
- is used.</li>
- </ul>
- <a name="user-content-page-rendering"/>
- <h3><a id="user-content-page-rendering" class="anchor" aria-hidden="true" href="#page-rendering"><svg class="octicon octicon-link" viewbox="0 0 16 16" version="1.1" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"/></svg></a>Page Rendering</h3>
- <ul>
- <li>Create a 'Frame Tree' or 'Render Tree' by traversing the DOM nodes, and
- calculating the CSS style values for each node.</li>
- <li>Calculate the preferred width of each node in the 'Frame Tree' bottom up
- by summing the preferred width of the child nodes and the node's
- horizontal margins, borders, and padding.</li>
- <li>Calculate the actual width of each node top-down by allocating each node's
- available width to its children.</li>
- <li>Calculate the height of each node bottom-up by applying text wrapping and
- summing the child node heights and the node's margins, borders, and padding.</li>
- <li>Calculate the coordinates of each node using the information calculated
- above.</li>
- <li>More complicated steps are taken when elements are <code>floated</code>,
- positioned <code>absolutely</code> or <code>relatively</code>, or other complex features
- are used. See
- <a href="http://dev.w3.org/csswg/css2/" rel="nofollow">http://dev.w3.org/csswg/css2/</a> and <a href="http://www.w3.org/Style/CSS/current-work" rel="nofollow">http://www.w3.org/Style/CSS/current-work</a>
- for more details.</li>
- <li>Create layers to describe which parts of the page can be animated as a group
- without being re-rasterized. Each frame/render object is assigned to a layer.</li>
- <li>Textures are allocated for each layer of the page.</li>
- <li>The frame/render objects for each layer are traversed and drawing commands
- are executed for their respective layer. This may be rasterized by the CPU
- or drawn on the GPU directly using D2D/SkiaGL.</li>
- <li>All of the above steps may reuse calculated values from the last time the
- webpage was rendered, so that incremental changes require less work.</li>
- <li>The page layers are sent to the compositing process where they are combined
- with layers for other visible content like the browser chrome, iframes
- and addon panels.</li>
- <li>Final layer positions are computed and the composite commands are issued
- via Direct3D/OpenGL. The GPU command buffer(s) are flushed to the GPU for
- asynchronous rendering and the frame is sent to the window server.</li>
- </ul>
- <a name="user-content-gpu-rendering"/>
- <h3><a id="user-content-gpu-rendering" class="anchor" aria-hidden="true" href="#gpu-rendering"><svg class="octicon octicon-link" viewbox="0 0 16 16" version="1.1" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"/></svg></a>GPU Rendering</h3>
- <ul>
- <li>During the rendering process the graphical computing layers can use general
- purpose <code>CPU</code> or the graphical processor <code>GPU</code> as well.</li>
- <li>When using <code>GPU</code> for graphical rendering computations the graphical
- software layers split the task into multiple pieces, so it can take advantage
- of <code>GPU</code> massive parallelism for float point calculations required for
- the rendering process.</li>
- </ul>
- <a name="user-content-window-server"/>
- <h3><a id="user-content-window-server" class="anchor" aria-hidden="true" href="#window-server"><svg class="octicon octicon-link" viewbox="0 0 16 16" version="1.1" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"/></svg></a>Window Server</h3>
- <a name="user-content-post-rendering-and-user-induced-execution"/>
- <h3><a id="user-content-post-rendering-and-user-induced-execution" class="anchor" aria-hidden="true" href="#post-rendering-and-user-induced-execution"><svg class="octicon octicon-link" viewbox="0 0 16 16" version="1.1" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"/></svg></a>Post-rendering and user-induced execution</h3>
- <p>After rendering has completed, the browser executes JavaScript code as a result
- of some timing mechanism (such as a Google Doodle animation) or user
- interaction (typing a query into the search box and receiving suggestions).
- Plugins such as Flash or Java may execute as well, although not at this time on
- the Google homepage. Scripts can cause additional network requests to be
- performed, as well as modify the page or its layout, causing another round of
- page rendering and painting.</p>
-
- </article>
|