lecture_02_architecture_large.pdf

# Web architecture and the HTTP protocol This section outlines the fundamental client-server model of web architectures, detailing the process of fetching a webpage and the structure and methods of HTTP requests and responses [2](#page=2). ### 1.1 Client-server web architecture The web operates on a client-server model, where clients (typically web browsers) request resources from servers. When a user types a URL like `http://www.vub.be` into their browser, a series of steps is initiated [2](#page=2): 1. **DNS lookup:** The Domain Name Service (DNS) is queried to translate the human-readable domain name (`www.vub.be`) into its corresponding IP address (e.g., `134.184.255.9`) [2](#page=2). 2. **TCP connection:** A Transmission Control Protocol (TCP) connection is established to the server's IP address [2](#page=2). 3. **HTTP request:** An HTTP request message is sent over the established TCP connection to the server [2](#page=2). 4. **HTTP response and visualization:** The server processes the request, fetches the requested resource, and sends back an HTTP response message, which the browser then renders [2](#page=2) [3](#page=3). ### 1.2 Web server tasks A web server is responsible for handling incoming HTTP requests and delivering resources. Its core tasks include [3](#page=3): * Establishing a connection with the client [3](#page=3). * Receiving and processing HTTP requests [3](#page=3). * Fetching the requested resource (e.g., an HTML file, image) [3](#page=3). * Creating and sending an HTTP response message back to the client [3](#page=3). * Logging requests and responses for auditing and analysis [3](#page=3). Prominent web servers include nginx, IIS, and the Apache HTTP Server. Many devices, such as printers, WLAN routers, and TVs, also feature embedded web servers [3](#page=3). ### 1.3 The HTTP protocol HTTP (Hypertext Transfer Protocol) is the foundation of data communication for the World Wide Web [6](#page=6). * **Request/response model:** It operates on a request/response communication model, where clients initiate all communication [6](#page=6). * **Stateless nature:** HTTP is a stateless protocol, meaning it does not inherently maintain session information between requests [6](#page=6). * **Transport layer:** HTTP can run over various reliable transport protocols, with TCP being the most common. It typically uses TCP port 80 by default [6](#page=6). * **HTTPS:** The HTTPS scheme signifies encrypted connections, providing security through protocols like SSL/TLS [6](#page=6). ### 1.4 HTTP message format HTTP messages, both requests and responses, share a common format consisting of a start line, header fields, a blank line, and an optional message body [8](#page=8). The general structure can be represented as: `HTTP_message = start_line, {header}, "CRLF", {body};` [8](#page=8). * **Start line:** The first line of an HTTP message, which is specific to either a request or a response [10](#page=10) [9](#page=9). * **Header fields:** A collection of key-value pairs providing metadata about the message [8](#page=8). * **Blank line:** A Carriage Return Line Feed (CRLF) separates the headers from the message body [8](#page=8). * **Message body:** Contains the actual data being transferred (e.g., HTML content, form data), and is optional [8](#page=8). > **Tip:** The `CRLF` is crucial as it signifies the end of the header section and the beginning of the body, or the end of the message if there is no body [8](#page=8). ### 1.5 HTTP request messages An HTTP request message is sent by the client to the server [9](#page=9). * **Start line:** The request-specific start line includes the method, the requested resource, and the HTTP version. The general format is [9](#page=9): `start_line = method, " ", resource, " ", version;` [9](#page=9). * **Methods:** These indicate the action the client wants to perform on the server's resource. Common methods include [9](#page=9): * `GET`: Retrieve a resource [9](#page=9). * `HEAD`: Retrieve only the headers of a resource [9](#page=9). * `POST`: Send data to the server, typically for creating or updating a resource [9](#page=9). * `PUT`: Upload a resource to the server [9](#page=9). * `TRACE`: Perform a message loop-back test [9](#page=9). * `OPTIONS`: Query the server about the communication options it supports [9](#page=9). * `DELETE`: Request the deletion of a resource [9](#page=9). * **Resource:** This can be a complete URL or a path to the resource on the server [9](#page=9). * **Version:** Specifies the HTTP protocol version, such as `HTTP/1.1` [9](#page=9). > **Example:** A `GET` request for the root of a website might look like: > ``` > GET / HTTP/1.1 > Host: www.example.com > ``` > [4](#page=4) [9](#page=9). ### 1.6 HTTP response messages An HTTP response message is sent by the server back to the client [10](#page=10). * **Start line:** The response-specific start line includes the HTTP version, a status code, and a reason phrase. The general format is [10](#page=10): `start_line = version, status_code, reason;` [10](#page=10). * **Status codes:** These codes indicate the outcome of the request. They are categorized as follows [10](#page=10): * `1xx` (Informational): The request was received and is continuing the process [10](#page=10). * `2xx` (Success): The action was successfully received, understood, and accepted (e.g., `200 OK`) [10](#page=10). * `3xx` (Redirection): Further action needs to be taken by the client to complete the request [10](#page=10). * `4xx` (Client Error): The request contains bad syntax or cannot be fulfilled (e.g., `404 Not Found`) [10](#page=10). * `5xx` (Server Error): The server failed to fulfill a valid request (e.g., `503 Service Unavailable`) [10](#page=10). * **Version:** The HTTP protocol version used by the server [10](#page=10). * **Reason:** A short text phrase describing the status code (e.g., "OK", "Not Found") [10](#page=10). > **Example:** A successful response might start with: > ``` > HTTP/1.1 200 OK > ``` > [10](#page=10) [5](#page=5). ### 1.7 HTTP header fields Header fields provide additional information about the request or response. They can be categorized into general headers, request headers, response headers, entity headers, and extension headers [11](#page=11) [8](#page=8). Key header fields include: * **`Accept` (Request Header):** Specifies the media types (MIME types) the client can understand [11](#page=11). * **`User-Agent` (Request Header):** Identifies the client software (e.g., browser) making the request [11](#page=11). * **`Keep-Alive` / `Persistent` (General Header):** Used in HTTP/1.0 and HTTP/1.1 respectively to enable persistent connections, reducing the overhead of establishing a new connection for each element on a webpage [11](#page=11). * **`Content-Type` (Entity Header):** Indicates the MIME type of the message body [11](#page=11). * **`If-Modified-Since` (Request Header):** Used with `GET` requests (conditional GET) to fetch a resource only if it has been modified since a specified date [12](#page=12). * **`Expires` (Response Header):** Specifies an expiration date after which a cached resource is considered stale [17](#page=17). * **`Cache-Control` (Response Header):** Provides directives for caching mechanisms, such as `max-age` (maximum age in seconds) or `no-cache` (requiring revalidation before serving from cache) [17](#page=17). > **Tip:** Understanding common HTTP headers is crucial for debugging network requests and optimizing web performance [11](#page=11) [12](#page=12) [17](#page=17). ### 1.8 Developers tools and testing HTTP Various developer tools can be used to inspect HTTP messages, such as the developer console in web browsers (e.g., Chrome's F12 or Ctrl+Shift+I). Simple telnet connections can also be used to manually send HTTP requests and observe responses [14](#page=14). > **Example:** Using telnet to send an HTTP request: > ```bash > telnet wise.vub.ac.be 80 > GET /beat-signer HTTP/1.1 > Host: wise.vub.ac.be > ``` > [14](#page=14). HTTP/2.0, introduced in May 2015 and inspired by Google's SPDY development, offers performance enhancements over HTTP/1.1 [14](#page=14). ### 1.9 Proxies A web proxy server acts as an intermediary between a client and a server, serving as a server to the client and a client to the server. Proxies are used for [15](#page=15): * **Firewalls and content filtering:** To control access to websites [15](#page=15). * **Transcoding:** On-the-fly transformation of HTTP message bodies, for example, to adapt content for different devices [15](#page=15). * **Content routing:** To select the optimal server in content distribution networks [15](#page=15). * **Anonymous browsing:** To mask the client's IP address [15](#page=15). ### 1.10 Caches A proxy cache is a specialized proxy server that stores frequently accessed resources to reduce server load and latency [16](#page=16). * **Caching hierarchies:** Caches are often organized in multi-level hierarchies (e.g., continental, national, regional) [16](#page=16). * **Cache types:** Caches can be passive (storing responses) or active (prefetching resources) [16](#page=16). * **HTTP cache control:** Special HTTP header fields like `Expires`, `Cache-Control`, and validators (`Last-modified`, `ETag`) manage caching behavior [17](#page=17). * **Advantages:** Caching reduces latency, network bandwidth usage, and server load. It is often transparent to the client and server [18](#page=18). * **Disadvantages:** Caching requires additional hardware resources. It can lead to serving stale data if not managed properly. Servers may lose control over access statistics as not all requests reach them directly [18](#page=18). ### 1.11 Tunnelling Tunnelling involves encapsulating one protocol within another to transmit it. A common use case is transmitting SSL/TLS connections (HTTPS) inside HTTP requests, which can help bypass firewalls that might otherwise block direct SSL connections [19](#page=19). > **Example:** Tunneling an SSL connection through an open HTTP port: > `HTTP[SSL]` [19](#page=19). ### 1.12 Gateways A gateway acts as an intermediary to translate or connect different applications and protocols [20](#page=20). * **Protocol translation:** Gateways can translate between protocols, such as from HTTP to FTP [20](#page=20). * **Security acceleration:** They can provide security functions, like handling SSL/TLS encryption on the server side (HTTPS to HTTP translation) [20](#page=20). * **Application integration:** Often, a gateway and the destination server are combined into a single application server that translates HTTP requests to server application commands [20](#page=20). > **Example:** An HTTP/FTP Gateway allowing an HTTP client to access an FTP server [20](#page=20). --- # Resource identification and management This topic explores how resources are identified using Uniform Resource Identifiers (URIs) and how web content is managed through mechanisms like caching and session management. ### 2.1 Uniform Resource Identifier (URI) A Uniform Resource Identifier (URI) is a string that uniquely identifies a resource. URIs are broadly categorized into two types: Uniform Resource Locators (URLs) and Uniform Resource Names (URNs) [7](#page=7). #### 2.1.1 Uniform Resource Locator (URL) A URL provides information about the exact location of a resource. It typically consists of a scheme, a host, and a path (the resource name). For example, `https://vub.academia.edu/BeatSigner` is a URL. A significant drawback of URLs is that they change if the resource is moved. Persistent Uniform Resource Locators (PURLs) are a variation designed to address this issue [7](#page=7). #### 2.1.2 Uniform Resource Name (URN) A URN serves as a unique and location-independent name for a resource. A URN is structured with a scheme name, a namespace identifier, and a namespace-specific string, all separated by colons. An example of a URN is `urn:ISBN:978-3837027136` [7](#page=7). ### 2.2 Web content management mechanisms Effective management of web content involves techniques to optimize retrieval and maintain state across stateless connections. #### 2.2.1 Caching Caching is a crucial mechanism for improving web performance by storing copies of resources closer to the user. HTTP provides several header fields to control caching behavior [17](#page=17). * **Expires**: Specifies an expiration date after which a cached resource must be refetched [17](#page=17). * **Cache-Control: max-age**: Defines the maximum age of a document in seconds from the time it was added to the cache [17](#page=17). * **Cache-Control: no-cache**: Indicates that a response cannot be served directly from the cache and must be revalidated with the origin server first [17](#page=17). Mechanisms known as validators are used to determine if cached content is still valid: * **Last-modified time**: When a cache holds a resource last modified at time `t`, it can use an `If-Modified-Since t` request to check for updates [17](#page=17). * **Entity tags (ETag)**: Publishers change the ETag if the content has been altered. Clients can then use an `If-None-Match etag` request to check for changes [17](#page=17). > **Tip:** Proper use of cache control headers can significantly reduce server load and improve user experience by delivering content faster. #### 2.2.2 Session management HTTP is inherently a stateless protocol, meaning each request is independent and carries no knowledge of previous requests. To overcome this, various solutions exist for tracking user sessions (state) [21](#page=21): * **Use of IP address**: While simple, this method is unreliable as IP addresses are often not uniquely assigned to a single user [21](#page=21). * **Browser login**: This approach uses special HTTP authentication headers. After a successful login, the browser automatically sends user information with each subsequent request [21](#page=21). * **URL rewriting**: This technique involves appending session information directly to the URL in each request [21](#page=21). * **Hidden form fields**: Similar to URL rewriting, session information can be embedded within form data, either in the URL (for GET requests) or in the body (for POST requests) [21](#page=21). * **Cookies**: Cookies are a widely used method where the server stores a small piece of information on the client. This information is then sent back to the server with every request to the same server [21](#page=21). ##### 2.2.2.1 Cookies Cookies were introduced by Netscape in June 1994. A cookie is a piece of information assigned to a client upon their first visit to a website. They are typically structured as a list of `` pairs, often containing a unique identifier [22](#page=22). Cookies are sent from the server to the browser via `Set-Cookie` HTTP response headers. The browser then stores this information in a "cookie database" and sends it back to the same server with every subsequent access [22](#page=22). > **Example:** When you log into a website, the server might send a cookie with a unique session ID to your browser. On your next request, your browser includes this cookie, allowing the server to recognize you and maintain your logged-in state. However, cookies can raise potential privacy concerns. Persistent cookies with long lifetimes can track users over extended periods, and third-party cookies can be used for tracking users across different websites. Users have the ability to disable cookies in their browser settings [22](#page=22). --- # Dynamic web content generation Dynamic web content generation involves technologies that allow web pages to change or adapt based on user interaction, server-side logic, or other factors, moving beyond static HTML delivery. This is achieved through both server-side and client-side processing [25](#page=25). ### 3.1 Server-side processing technologies Server-side processing allows for content to be generated or modified on the web server before being sent to the client's browser [25](#page=25). #### 3.1.1 Common Gateway Interface (CGI) CGI was one of the earliest solutions for server-side processing, operating transparently to the user. When a request arrives for a specific CGI script (e.g., `/account.pl`), the web server forwards it to a program written in languages like Perl, Tcl, C, C++, or Java. This program processes the request and generates an answer, potentially including HTTP response headers [26](#page=26). > **Tip:** The core idea of CGI is that the web server starts a new process for each incoming request to execute the CGI program. ##### 3.1.1.1 CGI problems A significant drawback of CGI is that a new process must be started for every request. This leads to poor performance, especially if the CGI program needs to establish a new database connection for each request [27](#page=27). FastCGI was developed to address some of these issues by introducing persistent processes and process pools, improving performance by reusing processes and connections. Despite these improvements, CGI and FastCGI have largely been superseded by newer technologies like Java Servlets [27](#page=27). #### 3.1.2 Java Servlets Java Servlets are Java classes that extend the abstract `HTTPServlet` class. A servlet container (like Apache Tomcat) loads these servlets and forwards relevant HTTP requests to them for processing [28](#page=28). > **Tip:** Servlet containers can be integrated with web servers or function as standalone components [29](#page=29). ##### 3.1.2.1 Servlet life cycle and methods The life cycle of a servlet involves: * **Initialization:** The servlet is initialized once via the `init()` method [29](#page=29). * **Request Handling:** The `doGet()` and `doPost()` methods can be executed multiple times, each handling a different HTTP request [29](#page=29). * **Destruction:** Before a servlet is unloaded, the servlet container calls the `destroy()` method [29](#page=29). The main methods involved are `init(ServletConfig config)`, `destroy()`, `doGet(HttpServletRequest req, HttpServletResponse resp)`, and `doPost(HttpServletRequest req, HttpServletResponse resp)` [29](#page=29). ##### 3.1.2.2 Java Servlet Example ```java package org.vub.wise; import java.io.*; import java.util.Date; import javax.servlet.http.*; import javax.servlet.*; public class HelloWorldServlet extends HttpServlet { public void doGet (HttpServletRequest req, HttpServletResponse res) throws ServletException, IOException { PrintWriter out = res.getWriter(); out.println(""); out.println(""); out.println("The time is " + new Date().toString() + ""); out.println(""); out.close(); } } ``` #### 3.1.3 Jakarta Server Pages (JSP) A perceived drawback of Java Servlets is that the entire HTML page must be defined within the Java code, making collaboration between web designers and programmers difficult. Jakarta Server Pages (JSP) addresses this by allowing program code to be embedded within HTML pages using scriptlets and markup. These JSP documents are then either interpreted on-the-fly or compiled into Java Servlets by a container like Apache Tomcat. The JSP approach is comparable to technologies like PHP and Active Server Pages (ASP). It's important to note that Java Servlets often serve as an enabling technology for frameworks like JSP [31](#page=31). #### 3.1.4 Node.js Node.js enables server-side JavaScript, offering functionality similar to Java Servlets for handling requests, databases, and sessions. A key advantage is the ability to write an entire application in a single language, JavaScript. Although server-side and client-side code remain separate, Node.js includes a built-in web server, negating the need for external servers like Apache or Tomcat. It boasts high modularity, allowing the addition of packages via npm for extended functionality. Numerous frameworks (e.g., Express, Passport, Sequelize) and template engines (e.g., Jade, EJS) are available, along with HTTP utility methods for sessions and routing [34](#page=34). ### 3.2 Client-side processing technologies Client-side processing occurs within the user's web browser, allowing for interactive and dynamic user experiences without constant server communication [25](#page=25). #### 3.2.1 JavaScript JavaScript is an interpreted scripting language primarily used for client-side processing. Its functionality can be embedded directly within HTML documents or provided in separate files. JavaScript is commonly employed for [32](#page=32): * Validating user input, such as form data [32](#page=32). * Dynamically adding content to existing webpages [32](#page=32). * Handling browser events like `onLoad` and `onFocus` [32](#page=32). * Modifying parts of the original HTML document [32](#page=32). * Creating and managing cookies [32](#page=32). > **Important Note:** Java and JavaScript are entirely distinct programming languages with no relation other than their name [32](#page=32). ##### 3.2.1.1 JavaScript Example ```html ``` #### 3.2.2 Java Applets Java Applets are programs delivered as Java bytecode to the client side, running within a secure sandbox environment in the browser via a Java Virtual Machine (JVM). An applet must extend either the `Applet` or `JApplet` class [35](#page=35). ##### 3.2.2.1 Advantages of Java Applets * Users always have the most up-to-date version of the applet [35](#page=35). * Provides a high level of security for untrusted applets [35](#page=35). * Trusted, signed applets have access to the full Java API [35](#page=35). ##### 3.2.2.2 Disadvantages of Java Applets * Requires a browser Java plug-in to run [35](#page=35). * Advanced functionality, such as network connections to machines other than the source machine, is restricted to signed applets [35](#page=35). ##### 3.2.2.3 Evolution beyond Java Applets OpenWebStart has emerged as a replacement for Java Applets, running programs outside the browser. This approach mitigates security restrictions and browser compatibility issues. Examples of Physics and Math Applets can be found at http://www.falstad.com/mathphysics.html [36](#page=36). --- # Web technologies and frameworks This section introduces foundational web technologies like HTML and explores the purpose and benefits of web application frameworks. ### 4.1 Hypertext Markup Language (HTML) Hypertext Markup Language (HTML) is the dominant markup language used for creating webpages. It forms the fundamental structure of content displayed in a web browser [24](#page=24). A basic HTML document structure typically includes: * A `` declaration to define the document type. * An `` element that serves as the root of the HTML page. * A `` section containing meta-information about the document, such as the title that appears in the browser tab [24](#page=24). * A `` section that encloses the visible content of the webpage, such as text, images, and links [24](#page=24). > **Tip:** For a deeper understanding of HTML, further resources are available, including dedicated lectures and exercise sessions [24](#page=24). ### 4.2 Web application frameworks A web application framework is a specialized software framework designed to streamline and support the development of dynamic websites, web applications, web services, and web resources [37](#page=37). The primary goal of these frameworks is to reduce the development overhead associated with common tasks in web development. Many frameworks achieve this by providing pre-built libraries and tools for essential functionalities, such as [37](#page=37): * Database access [37](#page=37). * Templating systems [37](#page=37). * Session management [37](#page=37). Furthermore, web application frameworks often promote code reuse, leading to more efficient and maintainable development practices. There are numerous web application frameworks available, with various options being presented in subsequent learning materials [37](#page=37). --- ## Common mistakes to avoid - Review all topics thoroughly before exams - Pay attention to formulas and key definitions - Practice with examples provided in each section - Don't memorize without understanding the underlying concepts

| Term | Definition | |------|------------| | HTTP Request | A message sent from a client to a server to request a resource or to send data to the server. It includes a start line, header fields, a blank line, and an optional message body. | | HTTP Response | A message sent from a server to a client in reply to an HTTP request. It contains a start line indicating the protocol version and status code, header fields providing metadata, a blank line, and an optional message body. | | Client-Server Model | A distributed application structure that partitions tasks or workloads between providers of a resource or service, called servers, and service requesters, called clients. In web technologies, the browser is the client and the web server is the server. | | Domain Name Service (DNS) | A hierarchical and decentralized naming system for computers, services, or other resources connected to the Internet or a private network. It translates human-readable domain names (like www.vub.be) into machine-readable IP addresses. | | TCP Connection | A Transmission Control Protocol (TCP) connection is a reliable, ordered, and error-checked byte stream established between two devices on a network. It is used to ensure data is delivered correctly between the client and server. | | Uniform Resource Identifier (URI) | A string of characters that unambiguously identifies a particular resource. URIs are used to locate and interact with resources on the web. | | Uniform Resource Locator (URL) | A specific type of URI that not only identifies a resource but also specifies how to locate it, typically including the scheme (e.g., http, https), host, and path to the resource. | | Uniform Resource Name (URN) | A type of URI that provides a persistent, location-independent identifier for a resource. It aims to uniquely name a resource regardless of where it is located. | | Web Server | A server program that accepts requests from web clients (browsers) and responds by serving web pages and other content. Prominent examples include nginx, IIS, and Apache HTTP Server. | | Stateless Protocol | A protocol that does not store any information about past interactions. Each request from a client to a server is treated as an independent transaction, without relying on previous requests. HTTP is an example of a stateless protocol. | | MIME Type (Media Type) | A standard way to indicate the nature and format of a document, file, or assortment of bytes. It consists of a type and a subtype (e.g., text/html, image/jpeg) and is used by clients and servers to process content appropriately. | | Proxy Server | An intermediary server that acts as a gateway between a client and other servers. Proxies can provide services like caching, filtering, security, and anonymous browsing. | | Cache | A temporary storage area used to hold frequently accessed data to speed up future requests. Web caches can reduce server load and improve response times by storing copies of web resources. | | Session Management | Techniques used to maintain state information across multiple requests from a client in a stateless protocol like HTTP. Common methods include cookies, URL rewriting, and hidden form fields. | | Cookies | Small pieces of data sent from a website and stored on the user's computer by the user's web browser while the user is browsing. They are used to remember stateful information for the user, such as items in a shopping cart or login credentials. | | HTML | HyperText Markup Language, the standard markup language for documents designed to be displayed in a web browser. It defines the structure and content of web pages. | | Dynamic Web Content | Web content that is generated or modified in real-time, either on the server-side or client-side, in response to user interactions or other factors, rather than being fixed static content. | | Server-Side Processing | Code that is executed on the web server to generate web content. This can involve database interactions, business logic, and dynamic page creation before the content is sent to the client. | | Client-Side Processing | Code that is executed by the user's web browser on the client machine. JavaScript is a common language used for client-side processing, enabling interactive features and dynamic content updates without requiring a full page reload. | | Common Gateway Interface (CGI) | An early standard for running external programs in a web server environment. It defines how a web server communicates with an external application to process requests and generate responses. | | Java Servlets | Server-side Java programs that extend the capabilities of a server. They are typically used to handle requests, process data, and generate dynamic web content. | | Jakarta Server Pages (JSP) | A server-side technology that allows developers to create dynamic web pages by embedding Java code within HTML. JSP pages are compiled into Java Servlets for execution. | | JavaScript | A scripting language commonly used for client-side web development to add interactivity, dynamic content, and asynchronous behavior to web pages. It can also be used for server-side development with environments like Node.js. | | Node.js | A JavaScript runtime environment that allows developers to run JavaScript code on the server-side. It is used for building scalable network applications, web servers, and APIs. | | Java Applets | Small Java programs that can be embedded into a web page and executed within a web browser's Java Virtual Machine (JVM). They were historically used for adding rich functionality to web pages but have largely been superseded by other technologies. | | Web Application Framework | A software framework designed to support the development of dynamic websites, web applications, web services, and web resources. Frameworks aim to streamline common web development tasks by providing libraries and tools. |