Did you know that browsers attach any stored cookies associated with a domain to any request to the domain? Sounds helpful but this got me wondering... what else do browsers do that we don't know about?

Sure, there are a lot of things that general users don't really need to know about, or care about , but as a security engineer I want to know more. I like knowing what's behind the curtain.

What is a browser?

The browser, commonly referred to as a "client", is a software program whose purpose is to present a web resource to you.
It's primary function is to process and render HTML and display it to the user; which may include text, links, and references to images and other items, such as CSS (cascading style sheets) and JavaScript functions.

How does a browser work?

At a very simplified level, browsers work by requesting a resource specified by the user, from a server, to display said resource back to the user in the browser display window.

That's pretty straight forward, but there are a lot of things in the background that make this happen.

What is each step that happens when I request a URL?

When you decided to visit Dev.to and made the request in your browser there are 4 main steps that happen:

• Request
• Response
• Build
• Render

Each of these steps are performed multiple times during the time it takes for a page to be requested and you see it load in your browser. If we break down each step we get:

Request

1. You specify a site you wish to access via the browsers address bar, Dev.to, and click "send" or press the Enter key.
2. The browser parses the URL to find the protocol, host, port, and path.
   • If HTTP was specified, the browser forms a HTTP request and the same goes for any other protocol at this level; such as ftp, https, or other Application Layer protocols.
3. The browser will need to convert the URL into the correct corresponding IP address via a DNS lookup.
   1. The browser checks its local cache for a DNS (Domain Name System) record to find the corresponding IP address of dev.to.
   2. If browser doesn’t contain the record in its cache for dev.to, it makes a system call to underlying OS (Operating System) to fetch the server’s IP address, as the OS also maintains a cache of recent DNS queries.
      1. If the OS also doesn't have a record in its cache, the router cache is the next in line to ask if it has a DNS record for Dev.to.
      2. If the router also has no DNS record in their cache for Dev.to the ISP (Internet Service Provider) takes a shot.
         1. The ISP checks its cashes and if no DNS record is found it runs a recursive search to find the resource.
         2. If still there is not record, then the ISP askes other DNS servers and the process continues until the resource is found or an error is returned.
4. Now that we have the IP address for Dev.to the browser will determine the type of request: "simple" or "complex".
   • A "simple" request is one that uses a "GET", "POST", or "HEAD" request method.
   • If the request is "complex", such as a Cross-site request, a "pre-flight request" is required before the request is sent. For a "preflighted" request the browser will first send an HTTP request using the OPTIONS method to the resource on the other origin, in order to determine if the actual request is safe and allowed to be sent. (5)
5. One the request type is determined the Browser will initiate a connection with the corresponding server by opening a socket from the user’s computer to that IP address, on the port specified (most often port 80 for HTTP).
   1. For a TCP connection (the most common connection for HTTP (HyperText Transfer Protocol) requests), the Browser initiates the "3-way handshake" by first sending a "SYN packet" (synchronize) to the server, asking if the server is open for new connections.
   2. If the server has open ports that can accept/initiate new connections, the server will respond with an acknowledgment of the "SYN packet" using a "SYN/ACK packet".
   3. The client will receive the "SYN/ACK packet" from the server that will acknowledge it by sending an "ACK packet" (acknowledge) back.
6. Now that a connection is established the Browser will send an HTTP request (GET, POST, DELETE, etc) to the server according to the specification of HTTP protocol through the established TCP connection.
   • This request will also contain additional information such as Browser headers like the User-Agent header (to identify the browser), any cookies that might be in the browsers' cache that are associated with the corresponding domain, the Accept header (to specify what types of requests that it will accept), and connection headers asking it to keep the TCP connection alive for additional requests, if necessary.

Response

1. The server receiving the request then inspects the request (most often only the path), and the subsequent behavior depends on if the site is dynamic or static.
2. Once the behavior is determined the server will send the request to its appropriate "Response Handler" to read and generate an appropriate response.
   • This could be something like Apache, Nginx, or the Oracle Java Platform SE.
3. The "Response Handler" will determine if it is a static of dynamic request and handle it accordingly.
   • For a static request a web server will be able to rapidly produce the HTML/CSS/JS and/or binary files (jpg, mp4, etc.) to each visitor.
   • For a dynamic request the web server will process any static parts of the request like a normal static request and then pass any dynamic parts to its appropriate web framework such as Django, Ruby on Rails, or Express that will process the full request and gather the related content from its libraries.
4. Once the "Response Handler" has generated the appropriate response object it will give it back to the server to generate a response to the browser who sends it back to the browser.
   • The response from the server usually contains the requested web page as well as response headers for the status code, compression type (Content-Encoding), how to cache the page (Cache-Control), any cookies to set, privacy information, etc.

Build

1. The Browser receives the response from the server and will cache the response for quicker access next time the resource is requested.
2. The Browser will then use a few different components for interpreting the HTML page and obtaining the remaining images, flash files, JavaScript files, CSS files, audio, video, etc. to build a Document Object Model (DOM) tree of the HTML.
   • Some browsers do this by choosing to interpret and render the HTML file first and then requesting the extra objects at the same time and fill in objects as they are received.
   • Others browsers will wait for all objects to be received before it will render and display the HTML file as a whole.

Render

1. Once the browser has all the parts of the website built in a DOM tree it will then use a few other components for rendering the DOM tree on the screen.
2. The Dev.to page you requested appears in your browser window.

Wrap up

The crazy part about this is how quick it happens and how slow it used to be. I grew up in the days of dial-up Internet so to see it grow to this lightning speed is fantastic and mind blowing how much happens in the literal blink of an eye.

Ok. I'm sure I missed a few things but I'll add them later. All in all I hope this helped make the process is a bit clearer. Toon in later for more about browsers =D

Happy Hacking

Resources:

1. https://www.html5rocks.com/en/tutorials/internals/howbrowserswork/#The_browsers_we_will_talk_about
2. https://www.liquidweb.com/kb/understanding-the-dns-process/
3. https://medium.com/@monica1109/what-happens-when-i-type-any-url-in-the-browser-3719c6357da2
4. https://www.toobler.com/from-http-request-to-dynamically-rendering-a-webpage/
5. https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS

Please Note: that I am still learning and if something that I have stated is incorrect please let me know. I would love to learn more about what I may not understand fully.