Understanding the Basics of How the Web Works
Oloruntobi Ajayi
Posted on April 24, 2024
A. Introduction
- Brief overview of the internet and its importance in modern society.
- Introduction to the World Wide Web (WWW) and its role in facilitating information exchange.
- Client-Server Architecture I will also discuss briefly what a client and server is. A Client is either a person or an organization using as a service. In the IT context, the client is a computer/device, also called a Host, that actually uses the service or accepts the information. Client devices include laptops, workstations, IoT devices, and similar network-friendly devices.
A Server in the IT world is a remote computer that provides access to data and services. Servers are usually physical devices such as rack servers, though the rise of cloud computing has brought virtual servers into the equation. The server handles processes like e-mail, application hosting, Internet connections, printing, and more.
Client-Server Architecture Explained
The client-server architecture refers to a system that hosts, delivers, and manages most of the resources and services that the client requests. In this model, all requests and services are delivered over a network, and it is also referred to as the networking computing model or client server network.
Client-server architecture, alternatively called a client-server model, is a network application that breaks down tasks and workloads between clients and servers that reside on the same system or are linked by a computer network.
Client-server architecture typically features multiple users’ workstations, PCs, or other devices, connected to a central server via an Internet connection or other network. The client sends a request for data, and the server accepts and accommodates the request, sending the data packets back to the user who needs them.
This model is also called a client-server network or a network computing model.
To sum it up briefly:
First, the client sends their request via a network-enabled device
Then, the network server accepts and processes the user request
Finally, the server delivers the reply to the client.
Client-server architecture typically features the following characteristics:
Client and server machines typically require different hardware and software resources and come from other vendors.
The network has horizontal scalability, which increases the number of client machines and vertical scalability, an then moves the entire process to more powerful servers or a multi-server configuration.
One computer server can provide multiple services simultaneously, although each service requires a separate server program.
Both client and server applications interact directly with a transport layer protocol. This process establishes communication and enables the entities to send and receive information.
Both the client and server computers need a complete stack of protocols. The transport protocol employs lower-layer protocols to send and receive individual messages.
- HTTP Protocol The Hypertext Transfer Protocol (HTTP) is the foundation of the World Wide Web, and is used to load webpages using hypertext links. HTTP is an application layer protocol designed to transfer information between networked devices and runs on top of other layers of the network protocol stack. A typical flow over HTTP involves a client machine making a request to a server, which then sends a response message. An HTTP request is the way Internet communications platforms such as web browsers ask for the information they need to load a website.
Each HTTP request made across the Internet carries with it a series of encoded data that carries different types of information. A typical HTTP request contains:
1.HTTP version type
2.a URL
3.an HTTP method
4.HTTP request headers
5.Optional HTTP body.
An HTTP method, sometimes referred to as an HTTP verb, indicates the action that the HTTP request expects from the queried server. For example, two of the most common HTTP methods are ‘GET’ and ‘POST’;
A ‘GET’ request expects information back in return (usually in the form of a website), while:
A ‘POST’ request typically indicates that the client is submitting information to the web server (such as form information, e.g. a submitted username and password).
HTTP headers contain text information stored in key-value pairs, and they are included in every HTTP request (and response, more on that later). These headers communicate core information, such as what browser the client is using and what data is being requested.
Example of HTTP request headers from Google Chrome's network tab:
HTTP request headers
What is in an HTTP request body?
The body of a request is the part that contains the ‘body’ of information the request is transferring. The body of an HTTP request contains any information being submitted to the web server, such as a username and password, or any other data entered into a form.
What is in an HTTP response?
An HTTP response is what web clients (often browsers) receive from an Internet server in answer to an HTTP request. These responses communicate valuable information based on what was asked for in the HTTP request.
A typical HTTP response contains:
an HTTP status code
HTTP response headers
optional HTTP body
Let's break these down:
What’s an HTTP status code?
HTTP status codes are 3-digit codes most often used to indicate whether an HTTP request has been successfully completed. Status codes are broken into the following 5 blocks:
1xx Informational
2xx Success
3xx Redirection
4xx Client Error
5xx Server Error
The “xx” refers to different numbers between 00 and 99.
Status codes starting with the number ‘2’ indicate a success. For example, after a client requests a webpage, the most commonly seen responses have a status code of ‘200 OK’, indicating that the request was properly completed.
If the response starts with a ‘4’ or a ‘5’ that means there was an error and the webpage will not be displayed. A status code that begins with a ‘4’ indicates a client-side error (it is very common to encounter a ‘404 NOT FOUND’ status code when making a typo in a URL). A status code beginning in ‘5’ means something went wrong on the server side. Status codes can also begin with a ‘1’ or a ‘3’, which indicate an informational response and a redirect, respectively.
What are HTTP response headers?
Much like an HTTP request, an HTTP response comes with headers that convey important information such as the language and format of the data being sent in the response body.
Example of HTTP response headers from Google Chrome's network tab:
HTTP response headers
What is in an HTTP response body?
Successful HTTP responses to ‘GET’ requests generally have a body which contains the requested information. In most web requests, this is HTML data that a web browser will translate into a webpage.
Can DDoS attacks be launched over HTTP?
Keep in mind that HTTP is a “stateless” protocol, which means that each command runs independent of any other command. In the original spec, HTTP requests each created and closed a TCP connection. In newer versions of the HTTP protocol (HTTP 1.1 and above), persistent connection allows for multiple HTTP requests to pass over a persistent TCP connection, improving resource consumption. In the context of DoS or DDoS attacks, HTTP requests in large quantities can be used to mount an attack on a target device, and are considered part of application layer attacks or layer 7 attacks.
- DNS (Domain Name System) Resolution The Domain Name System (DNS) is the phonebook of the Internet. Humans access information online through domain names, like nytimes.com or espn.com. Web browsers interact through Internet Protocol (IP) addresses. DNS translates domain names to IP addresses so browsers can load Internet resources.
Each device connected to the Internet has a unique IP address which other machines use to find the device. DNS servers eliminate the need for humans to memorize IP addresses such as 192.168.1.1 (in IPv4), or more complex newer alphanumeric IP addresses such as 2400:cb00:2048:1::c629:d7a2 (in IPv6).
How does DNS work?
The process of DNS resolution involves converting a hostname (such as www.example.com) into a computer-friendly IP address (such as 192.168.1.1). An IP address is given to each device on the Internet, and that address is necessary to find the appropriate Internet device - like a street address is used to find a particular home. When a user wants to load a webpage, a translation must occur between what a user types into their web browser (example.com) and the machine-friendly address necessary to locate the example.com webpage.
In order to understand the process behind the DNS resolution, it’s important to learn about the different hardware components a DNS query must pass between. For the web browser, the DNS lookup occurs "behind the scenes" and requires no interaction from the user’s computer apart from the initial request.
There are 4 DNS servers involved in loading a webpage:
1.DNS recursor - The recursor can be thought of as a librarian who is asked to go find a particular book somewhere in a library. The DNS recursor is a server designed to receive queries from client machines through applications such as web browsers. Typically the recursor is then responsible for making additional requests in order to satisfy the client’s DNS query.
2.Root nameserver - The root server is the first step in translating (resolving) human readable host names into IP addresses. It can be thought of like an index in a library that points to different racks of books - typically it serves as a reference to other more specific locations.
3.TLD nameserver - The top level domain server (TLD) can be thought of as a specific rack of books in a library. This nameserver is the next step in the search for a specific IP address, and it hosts the last portion of a hostname (In example.com, the TLD server is “com”).
4.Authoritative nameserver - This final nameserver can be thought of as a dictionary on a rack of books, in which a specific name can be translated into its definition. The authoritative nameserver is the last stop in the nameserver query. If the authoritative name server has access to the requested record, it will return the IP address for the requested hostname back to the DNS Recursor (the librarian) that made the initial request.
What's the difference between an authoritative DNS server and a recursive DNS resolver?
Both concepts refer to servers (groups of servers) that are integral to the DNS infrastructure, but each performs a different role and lives in different locations inside the pipeline of a DNS query. One way to think about the difference is the recursive resolver is at the beginning of the DNS query and the authoritative nameserver is at the end.
Recursive DNS resolver
The recursive resolver is the computer that responds to a recursive request from a client and takes the time to track down the DNS record. It does this by making a series of requests until it reaches the authoritative DNS nameserver for the requested record (or times out or returns an error if no record is found). Luckily, recursive DNS resolvers do not always need to make multiple requests in order to track down the records needed to respond to a client; caching is a data persistence process that helps short-circuit the necessary requests by serving the requested resource record earlier in the DNS lookup.
DNS Record Request Sequence - DNS Recursive Resolver gets request from client
Authoritative DNS server
Put simply, an authoritative DNS server is a server that actually holds, and is responsible for, DNS resource records. This is the server at the bottom of the DNS lookup chain that will respond with the queried resource record, ultimately allowing the web browser making the request to reach the IP address needed to access a website or other web resources. An authoritative nameserver can satisfy queries from its own data without needing to query another source, as it is the final source of truth for certain DNS records.
DNS Record Request Sequence - DNS query reaches authoritative nameserver for cloudflare.com
It’s worth mentioning that in instances where the query is for a subdomain such as foo.example.com or blog.cloudflare.com, an additional nameserver will be added to the sequence after the authoritative nameserver, which is responsible for storing the subdomain’s CNAME record.
DNS Record Request Sequence - DNS query to CNAME record for subdomain blog.cloudflare.com
There is a key difference between many DNS services and the one that Cloudflare provides. Different DNS recursive resolvers such as Google DNS, OpenDNS, and providers like Comcast all maintain data center installations of DNS recursive resolvers. These resolvers allow for quick and easy queries through optimized clusters of DNS-optimized computer systems, but they are fundamentally different than the nameservers hosted by Cloudflare.
Cloudflare maintains infrastructure-level nameservers that are integral to the functioning of the Internet. One key example is the f-root server network which Cloudflare is partially responsible for hosting. The F-root is one of the root level DNS nameserver infrastructure components responsible for the billions of Internet requests per day. Our Anycast network puts us in a unique position to handle large volumes of DNS traffic without service interruption.
What are the steps in a DNS lookup?
For most situations, DNS is concerned with a domain name being translated into the appropriate IP address. To learn how this process works, it helps to follow the path of a DNS lookup as it travels from a web browser, through the DNS lookup process, and back again. Let's take a look at the steps.
Note: Often DNS lookup information will be cached either locally inside the querying computer or remotely in the DNS infrastructure. There are typically 8 steps in a DNS lookup. When DNS information is cached, steps are skipped from the DNS lookup process which makes it quicker. The example below outlines all 8 steps when nothing is cached.
The 8 steps in a DNS lookup:
1.A user types ‘example.com’ into a web browser and the query travels into the Internet and is received by a DNS recursive resolver.
2.The resolver then queries a DNS root nameserver (.).
3.The root server then responds to the resolver with the address of a Top Level Domain (TLD) DNS server (such as .com or .net), which stores the information for its domains. When searching for example.com, our request is pointed toward the .com TLD.
4.The resolver then makes a request to the .com TLD.
5.The TLD server then responds with the IP address of the domain’s nameserver, example.com.
6.Lastly, the recursive resolver sends a query to the domain’s nameserver.
7.The IP address for example.com is then returned to the resolver from the nameserver.
8.The DNS resolver then responds to the web browser with the IP address of the domain requested initially.
Once the 8 steps of the DNS lookup have returned the IP address for example.com, the browser is able to make the request for the web page:
The browser makes a HTTP request to the IP address.
The server at that IP returns the webpage to be rendered in the browser (step 10).
Complete DNS Lookup and Webpage Query - 10 steps
What is a DNS resolver?
The DNS resolver is the first stop in the DNS lookup, and it is responsible for dealing with the client that made the initial request. The resolver starts the sequence of queries that ultimately leads to a URL being translated into the necessary IP address.
Note: A typical uncached DNS lookup will involve both recursive and iterative queries.
It's important to differentiate between a recursive DNS query and a recursive DNS resolver. The query refers to the request made to a DNS resolver requiring the resolution of the query. A DNS recursive resolver is the computer that accepts a recursive query and processes the response by making the necessary requests.
DNS recursive query goes from DNS client to DNS recursive resolver
What are the types of DNS queries?
In a typical DNS lookup three types of queries occur. By using a combination of these queries, an optimized process for DNS resolution can result in a reduction of distance traveled. In an ideal situation cached record data will be available, allowing a DNS name server to return a non-recursive query.
3 types of DNS queries:
Recursive query - In a recursive query, a DNS client requires that a DNS server (typically a DNS recursive resolver) will respond to the client with either the requested resource record or an error message if the resolver can't find the record.
Iterative query - in this situation the DNS client will allow a DNS server to return the best answer it can. If the queried DNS server does not have a match for the query name, it will return a referral to a DNS server authoritative for a lower level of the domain namespace. The DNS client will then make a query to the referral address. This process continues with additional DNS servers down the query chain until either an error or timeout occurs.
Non-recursive query - typically this will occur when a DNS resolver client queries a DNS server for a record that it has access to either because it's authoritative for the record or the record exists inside of its cache. Typically, a DNS server will cache DNS records to prevent additional bandwidth consumption and load on upstream servers.
What is DNS caching? Where does DNS caching occur?
The purpose of caching is to temporarily stored data in a location that results in improvements in performance and reliability for data requests. DNS caching involves storing data closer to the requesting client so that the DNS query can be resolved earlier and additional queries further down the DNS lookup chain can be avoided, thereby improving load times and reducing bandwidth/CPU consumption. DNS data can be cached in a variety of locations, each of which will store DNS records for a set amount of time determined by a time-to-live (TTL).
Browser DNS caching
Modern web browsers are designed by default to cache DNS records for a set amount of time. The purpose here is obvious; the closer the DNS caching occurs to the web browser, the fewer processing steps must be taken in order to check the cache and make the correct requests to an IP address. When a request is made for a DNS record, the browser cache is the first location checked for the requested record.
Operating system (OS) level DNS caching
The operating system level DNS resolver is the second and last local stop before a DNS query leaves your machine. The process inside your operating system that is designed to handle this query is commonly called a “stub resolver” or DNS client. When a stub resolver gets a request from an application, it first checks its own cache to see if it has the record. If it does not, it then sends a DNS query (with a recursive flag set), outside the local network to a DNS recursive resolver inside the Internet service provider (ISP).
When the recursive resolver inside the ISP receives a DNS query, like all previous steps, it will also check to see if the requested host-to-IP-address translation is already stored inside its local persistence layer.
The recursive resolver also has additional functionality depending on the types of records it has in its cache:
1.If the resolver does not have the A record, but does have the NS records for the authoritative nameservers, it will query those name servers directly, bypassing several steps in the DNS query. This shortcut prevents lookups from the root and .com nameservers (in our search for example.com) and helps the resolution of the DNS query occur more quickly.
2.If the resolver does not have the NS records, it will send a query to the TLD servers (.com in our case), skipping the root server.
3.In the unlikely event that the resolver does not have records pointing to the TLD servers, it will then query the root servers. This event typically occurs after a DNS cache has been purged.
Browser Rendering Process
The browser rendering process refers to the sequence of steps involved in displaying a web page on a user's browser. It encompasses the interpretation of HTML, CSS, and JavaScript code to render a visually appealing and interactive web page. Here's an overview of the browser rendering process:-
HTML Parsing and DOM Construction:
- The browser receives the HTML document from the server.
- The HTML parser parses the HTML markup and constructs the Document Object Model (DOM) tree.
- The DOM tree represents the structure of the HTML document, with each element as a node in the tree.
-
CSS Parsing and Style Computation:
- As the HTML parser encounters
<style>
or<link>
elements referencing external or embedded CSS, it initiates the CSS parsing process. - The CSS parser parses the CSS rules and constructs the CSS Object Model (CSSOM).
- The browser combines the DOM tree and CSSOM to create the Render Tree.
- The Render Tree contains only the nodes that are visible on the web page and their computed styles.
- As the HTML parser encounters
-
Layout (or Reflow):
- The browser calculates the layout or geometry of each element in the Render Tree.
- It determines the position, size, and spacing of each element relative to its parent and sibling elements.
- Layout calculations consider factors like width, height, margins, padding, and positioning properties.
-
Painting:
- After layout calculations, the browser paints the pixels on the screen according to the computed styles and layout.
- The browser creates layers based on elements with certain properties (e.g.,
position: fixed
,transform
,opacity
) to optimize rendering. - Each layer is painted individually, and then composited to form the final visual representation of the web page.
-
JavaScript Execution:
- If the web page contains JavaScript code, the browser's JavaScript engine executes it.
- JavaScript can modify the DOM, CSSOM, or trigger asynchronous operations like fetching data from servers (AJAX requests).
- Changes made by JavaScript may require reflow and repaint, impacting performance if not optimized.
-
Event Handling and Interaction:
- Once the web page is rendered, the browser listens for user interactions such as clicks, scrolls, or keyboard inputs.
- Event handlers associated with DOM elements respond to user actions, triggering changes in the page's state or behavior.
The browser rendering process is complex and involves multiple stages, each crucial for delivering a seamless and interactive browsing experience to users. Efficient coding practices, optimization techniques, and understanding browser internals can help improve rendering performance and overall user satisfaction.
In conclusion, understanding how the web works involves grasping the intricate interplay between various technologies and protocols that enable the seamless exchange of information over the Internet. The client-server model forms the foundation of this ecosystem, with web browsers acting as the interface for users to access and interact with web content.
The HTTP protocol governs the communication between clients and servers, facilitating the transfer of resources such as HTML, CSS, JavaScript, images, and more. DNS plays a crucial role in translating human-readable domain names into IP addresses, allowing users to access websites using familiar URLs.
At the heart of the browsing experience lies the browser rendering process, where HTML, CSS, and JavaScript are parsed, interpreted, and rendered to create visually appealing and interactive web pages. This process involves constructing the DOM and CSSOM, calculating layouts, painting pixels, executing JavaScript, and handling user interactions.
By delving into the fundamentals of the web, including client-server architecture, the HTTP request-response cycle, DNS resolution, and browser rendering, beginners and seasoned developers alike can gain a deeper appreciation for the technologies shaping our online experiences. Armed with this knowledge, individuals can build and optimize web applications, ensuring efficiency, reliability, and user satisfaction in an ever-evolving digital landscape.
Posted on April 24, 2024
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.