COMP3331/9331 Computer Networks and Applications HTTP Implementation - Assignment for Term 2, 2025 Due: 16:59 Thursday, 31 July 2025 (Week 9) Date Description 29/05/2025 Initial release ? Useful tools for testing and debugging ? Frequently asked questions Introduction Background Task Scope Learning Objectives Resources Deliverables Interface HTTP Concepts Requests vs. Responses HTTP Methods Request Target Forms Response Status Codes Header Fields Message Body Message Parsing Requirements & Features Basic Non-Persistent Proxy Basic Persistent Proxy HTTP Tunneling with CONNECT Error Handling Explicit Error Conditions Caching Logging Concurrency Report Tips on Getting Started Testing and Debugging Submission Late Submission Policy Special Consideration and Equitable Learning Services 2025/7/18 17:54 Assignment Specification | COMP3331/9331 25T2 https://cgi.cse.unsw.edu.au/~cs3331/25T2/assign/index.html 1/25 Plagiarism Marking Rubric Introduction Background The Hypertext Transfer Protocol (HTTP) is the backbone of the World Wide Web, enabling everything from browsing websites and streaming videos to online banking, shopping, and social media. Every time you load a webpage, send a message, or access cloud-based applications, HTTP facilitates communication between your device and remote servers, making modern internet usage seamless and efficient. HTTP allows the use of intermediaries to handle requests through a chain of connections. One common type of intermediary is a proxy, a message-forwarding agent that sits between a client (such as a Web browser) and an origin server. Instead of connecting directly to the origin server, the client sends its request to the proxy, which then forwards it on behalf of the client. The origin server s response is then relayed back through the proxy to the client. Because a proxy handles both incoming and outgoing requests, it effectively acts as both a client and a server a server to the clients making requests and a client to the servers fulfilling them. Figure 1: The proxy as a middleman Proxies serve a variety of purposes, including: Filtering requests, such as blocking access to restricted websites. Transforming responses, such as compressing or converting images to reduce bandwidth usage. Caching responses, to improve performance and reduce redundant network traffic. Anonymising clients, by stripping identifying information from requests. Task In this assignment, you will implement your own HTTP/1.1 proxy in C, Java or Python. Your proxy should: Support the CONNECT, GET, HEAD, and POST methods. Handle multiple requests over a single connection (persistence). 2025/7/18 17:54 Assignment Specification | COMP3331/9331 25T2 https://cgi.cse.unsw.edu.au/~cs3331/25T2/assign/index.html 2/25 Handle multiple client connections concurrently (concurrency). Cache responses for potential reuse in future requests. Your proxy must compile and run within the CSE environment. Ensure it is thoroughly tested in that environment. You are only permitted to use the basic libraries for socket programming. You must not use any ready-made server or HTTP libraries to implement any aspects of this assignment. Doing so will likely result in a mark of zero. If in doubt, please check with course staff on the forum. This is an individual assignment and is worth 20 marks. Scope This assignment specification is the authoritative reference for your implementation. While it is based on HTTP standards, to simplify your task certain aspects may deviate from or contradict official specifications. In such cases, the requirements outlined here take precedence. If you encounter any ambiguity, seek clarification via the course forum rather than relying on official specifications. Learning Objectives By completing this assignment, you will gain a deeper understanding of: HTTP as an application-layer protocol and its role in web communication. The client-server model and how intermediaries like proxies interact with clients and servers. TCP socket programming for handling network communication. Concurrent programming to manage multiple connections efficiently. Caching mechanisms to improve performance and reduce redundant requests. Resources Lecture Notes: Application Layer Part 1 C slides on HTTP and Web caches See Moodle for the lecture recording Related textbook chapter: 2.2 The Web and HTTP Lecture Notes: Application Layer Part 2 C slides on socket programming See Moodle for the lecture recording Related textbook chapter: 2.7 Socket Programming: Creating Network Applications Sample Client-Server Programs and Networking Programming Resources Lab 2 C HTTP and Socket Programming Lab 3 C A Simple Web Server Frequently Asked Questions Useful Tools for Testing and Debugging Programming Tutorial - in week 7, more details to come Help Sessions - starting week 7, more details to come MDN Web Docs C a great resource for web technologies RFC 9110 C HTTP Semantics (optional, for those interested) RFC 9111 C HTTP Caching (optional, for those interested) RFC 9112 C HTTP/1.1 (optional, for those interested) 2025/7/18 17:54 Assignment Specification | COMP3331/9331 25T2 https://cgi.cse.unsw.edu.au/~cs3331/25T2/assign/index.html 3/25 Deliverables Source code The main entry point of your program must be named one of: proxy.c (if implementing in C) Proxy.java (if implementing in Java) proxy.py (if implementing in Python) You are free to create additional helper files and name them as you wish. A Makefile (not required for Python) Running make should produce an executable named: proxy (if implementing in C) Proxy.class (if implementing in Java) A Report named report.pdf Interface Your proxy must accept 4 command-line arguments: port: the TCP port the proxy will listen on for client connections. We recommend using a random port number between 49152 and 65535 (the dynamic port number range). timeout: a strictly positive integer, in seconds. It's the duration a client or server may be idle or unresponsive. max_object_size: a strictly positive integer, in bytes. It's the maximum object size that can be cached. max_cache_size: a strictly positive integer, in bytes, and at least equal to max_object_size. It's the total amount of object data that may be cached. It should be initiated as follows: $ ./proxy
# for C $ java Proxy
# for
Java $ python3 proxy.py
# for
Python For example, running one of: $ ./proxy
60893 10 1024 1048576
# for C $ java Proxy
60893 10 1024 1048576
# for Java $ python3 proxy.py 60893 10 1024 1048576
# for Python Would mean the proxy should: Listen on port 60893 (avoid this port on CSE, other students will likely try and use it simply because it's in the spec!). Timeout clients/servers after 10 seconds. 2025/7/18 17:54 Assignment Specification | COMP3331/9331 25T2 https://cgi.cse.unsw.edu.au/~cs3331/25T2/assign/index.html 4/25 Not cache any objects greater than 1024 bytes. Not allow the cache to grow any greater than 1048576 bytes. You must ensure that none of these values are hard-coded. They must be configurable via the command-line arguments. HTTP Concepts Requests vs. Responses HTTP is a stateless request/response protocol used for exchanging messages over a network connection. These messages fall into two categories: requests, which clients send to initiate an action on the server, and responses, which servers return to fulfill those requests. Both types of messages share a common structure. They begin with a start-line, followed by zero or more header field lines (collectively called the headers or header section), an empty line marking the end of the headers, and an optional message body. The line terminator for the start-line and header fields is the sequence CRLF (\r\n, representing carriage return and line feed). Structurally, the only difference between request and response messages is the start-line: Requests: Include a request-line specifying the HTTP method, request target, and protocol version. Responses: Include a status-line with the protocol version, status code, and an optional reason phrase. POST / HTTP/1.1 Host: developer.mozilla.org User-Agent: curl/8.6.0 Accept: */* Content-Type: application/json Content-Length: 345 { "data": "ABC123" } HTTP/1.1 403 Forbidden Server: Apache Date: Fri, 21 Jun 2024 12:52:39 GMT Content-Length: 678 Content-Type: text/html Cache-Control: no-store
(more data ) Request Response Start line Headers Empty line Body Figure 2: Anatomy of an HTTP message (Source: MDN Web Docs) HTTP Methods The request method defines the purpose of the request and what the client expects as a successful result. GET: Requests a resource from the server. The request has no body, but the response tyically does. HEAD: Requests the headers (only) that would by returned if it were a GET request. Neither the request nor the response has a body. POST: Sends data to the server, often to create or update a resource. 2025/7/18 17:54 Assignment Specification | COMP3331/9331 25T2 https://cgi.cse.unsw.edu.au/~cs3331/25T2/assign/index.html 5/25 Both the request and the response typically have a body. CONNECT: Establishes a tunnel to a server, typically used for HTTPS over a proxy. The client sends CONNECT, and if successful, raw data flows between the client and the server. Neither the request nor the response has a body. Request Target Forms The request target specifies the resource on which the method should be applied. The form of the request target varies depending on the HTTP method and whether the request is sent through a proxy. Origin-Form Used in most requests sent directly to an origin server (e.g., when a client connects to a web server). The request target consists of just the path and optional query string. If the path component is empty, the client must send / as the path within the origin-form of a request-target. Example: GET /index.html?query=123 HTTP/1.1 Absolute-Form Used when a request is sent through a proxy (except for CONNECT requests). The request target includes the scheme (http), hostname, optional port, path, and query string. If a port is omitted, it defaults to port 80 for http. Example: HEAD http://example.com:8080/index.html?query=123 HTTP/1.1 Authority-Form Used exclusively by the CONNECT method when requesting a proxy to establish a tunnel to a remote server. The request target consists of only the hostname and port (without a scheme or path). The port must be included explicitly. If omitted, the request is invalid. Example: CONNECT example.com:443 HTTP/1.1 Response Status Codes 2025/7/18 17:54 Assignment Specification | COMP3331/9331 25T2 https://cgi.cse.unsw.edu.au/~cs3331/25T2/assign/index.html 6/25 HTTP response status codes indicate whether a specific HTTP request has been successfully completed. Responses are grouped into five classes: 1. Informational responses (100 C 199) - Note: your proxy will not encounter these during marking The request was received but still being processed. 2. Successful responses (200 C 299) The request was successfully received, understood, and accepted. 3. Redirection responses (300 C 399) Further action needs to be taken in order to complete the request. 4. Client error responses (400 C 499) The request contains bad syntax or cannot be fulfilled. 5. Server error responses (500 C 599) The server failed to fulfill an apparently valid request. Header Fields Header fields provide metadata about an HTTP message. They follow this format, where the field name and field value are colon separated: field-name: field-value Field names are case-insensitive, and there should be no leading or trailing whitespace. Field values themselves do not have any leading or trailing whitespace, however a field value may be surrounded by optional whitespace. Some header fields may be multi-valued, as an ordered, comma-separated list: field-name: field-value-1, field-value-2, ..., field-value-n As part of this assignment, before forwarding a message, your proxy will need to identify, and potentially remove, modify, or insert, particular header fields. These fields are briefly described here. Any other fields that may be present should simply be forwarded unchanged. 1. Connection / Proxy-Connection The Connection header allows the sender to list desired control options for the current connection. Most notably, it controls whether the network connection stays open (i.e. persists) after the current transaction finishes. If a sender does not intend to keep the connection open beyond the current request-response exchange, then they must specify a close directive: Connection: close 2025/7/18 17:54 Assignment Specification | COMP3331/9331 25T2 https://cgi.cse.unsw.edu.au/~cs3331/25T2/assign/index.html 7/25 Otherwise, they may specify a list of desired control options, for example: Connection: keep-alive, upgrade In HTTP/1.1, connections are persistent by default, meaning unless there's an explicit close directive in the message, the connection should remain open, regardless of whether there is a keep-alive directive. Some legacy user agents may send a non-standard Proxy-Connection header instead of or in addition to Connection, attempting to control connection persistence with proxies. This is not part of the official HTTP specification, but proxies should handle both headers appropriately. Effectively, they should be regarded as equivalent, except where both are present, in which case Connection takes precedence. Note, the header values, like the header name, are not case sensitive. 2. Transfer-Encoding The Transfer-Encoding header lists any transformations that have been applied to the content in order to form the message body. For example, the following header indicates that the content has been compressed in gzip format and that it will be sent as a series of chunks: Transfer-Encoding: gzip, chunked Chunked transfer encoding allows data to be sent in multiple parts, each prefixed with its size, without knowing the total size in advance. If a message is received with both a Transfer-Encoding and a Content-Length header, the Transfer-Encoding overrides the Content-Length. Note, the header values, like the header name, are not case sensitive. 3. Content-Length If a message doesn t have a Transfer-Encoding header, the Content-Length header can specify the expected size of the content in bytes. When a message includes content, Content-Length can help determine where the data and message end. For messages without content, it indicates the size of the selected representation. For example, a response to a HEAD request does not include content, but may include a Content-Length. 4. Via The Via header is used for tracking message forwards. The syntax is a comma-separated list of
pairs. 2025/7/18 17:54 Assignment Specification | COMP3331/9331 25T2 https://cgi.cse.unsw.edu.au/~cs3331/25T2/assign/index.html 8/25 For example, the following header indicates that the message passed through an intermediary that supports HTTP/1.0 with the pseudonym foo, then an intermediary that supports HTTP/1.1 with the pseudonym bar. Via: 1.0 foo, 1.1 bar
Message Body The message body (if any) is used to carry content for the request or response. The message body is identical to the content unless a transfer coding has been applied. The rules for determining when a message body is present in a message differ for requests and responses: Requests: The presence of a message body is signaled by a Content-Length header field. Responses: The presence of a message body depends on both the request method to which it is responding and the response status code. The length of a message body is determined by one of the following (in order of precedence): 1. Any response to a HEAD request and any response with a 204 (No Content) or 304 (Not Modified) status code is always terminated by the first empty line after the header fields, regardless of the header fields present in the message, and thus cannot contain a message body. 2. If a Transfer-Encoding header field is present in a response, the message body length is determined by reading the connection until it is closed by the server. Note: this is a significant deviation from HTTP/1.1 standards, made for the purpose of simplifying the assignment. 3. If a Content-Length header field is present without Transfer-Encoding, its decimal value defines the expected message body length in bytes. If the sender closes the connection or the recipient times out before the indicated number of bytes are received, the recipient must consider the message to be incomplete and close the connection. 4. If this is a request message and none of the above are true, then the message body length is zero (no message body is present). 5. Otherwise, this is a response message without a declared message body length, so the message body length is determined by the number of bytes received prior to the server closing the connection. Message Parsing As a general guide, when parsing an HTTP message, the typical approach is: 1. Read the start-line into a structured representation. 2. Read each header field line into a hash table (keyed by field name) or similar, until encountering an empty line. 3. Use the parsed data to determine whether a message body is expected. 4. If a message body is indicated, read it as a stream, either for the specified length or until the connection is closed. 2025/7/18 17:54 Assignment Specification | COMP3331/9331 25T2 https://cgi.cse.unsw.edu.au/~cs3331/25T2/assign/index.html 9/25 Requirements & Features Basic Non-Persistent Proxy In the simplest case, your proxy will be non-persistent and handle requests sequentially, supporting GET, HEAD, and POST. While it must accommodate multiple client connections over time, it only needs to handle one connection at a time, with each connection limited to a single request-response exchange. This is illustrated in the following example of a typical non-persistent transaction, starting with Figure 3. Important reminder: All header field names, and all header field values that we are dealing with directly, should be treated as case-insensitive. 1. Client-Proxy Connection The client initiates a connection with the proxy, which the proxy accepts. 2. Client Request The proxy should start receiving and parsing the request message, for example: Parse the start line to extract the method, request target, and protocol version. Parse the absolute-form request target to extract the hostname, optional port, and origin- form path + optional query. In this case, no port is given, so it defaults to port 80. Read each header field line into a data structure, until encountering an empty line, marking the end of the header section. Use the parsed data to determine whether a message body is expected and if more data may need to be read. In this case, the message is a request and the method is GET, so no body is expected and the message is considered complete. The proxy should then transform the message for forwarding, in particular: Replace the request target with its origin-form. Replace or insert any Connection header with Connection: close. Remove any Proxy-Connection header. Insert or append a Via header with 1.1 [your zID]. 3. Proxy-Server Connection The proxy initiates a connection with the origin server specified in the request target, which the origin server accepts. 4. Client Request Forwarding The proxy sends the transformed request to the origin server. 2025/7/18 17:54 Assignment Specification | COMP3331/9331 25T2 https://cgi.cse.unsw.edu.au/~cs3331/25T2/assign/index.html 10/25 Figure 3: A client sending a GET request via a proxy 5. Server Response The proxy should start receiving and parsing the response message, for example: Parse the start line to extract the protocol version, status code, and optional reason phrase. Read each header field line into a a data structure, until encountering an empty line. Determine whether a message body is expected. In this case, the message is in response to a GET request and has a status code of 200, therefore a body is expected. Determine that no Transfer-Encoding has been specified, but a Content-Length of 1256 bytes is indicated, so continue to read data if and as necessary until the full body is received. 6. Proxy-Server Connection Termination The proxy closes the connection with the server. 7. Server Response Forwarding The proxy should then transform the message for forwarding, in particular: If necessary, replace or insert any Connection header with Connection: close. Insert or append a Via header with 1.1 [your zID]. The proxy sends the transformed response to the client. 8. Client-Proxy Connection Termination The proxy closes the connection with the client and listens for the next connection request. 2025/7/18 17:54 Assignment Specification | COMP3331/9331 25T2 https://cgi.cse.unsw.edu.au/~cs3331/25T2/assign/index.html 11/25 Figure 4: A server sending a response to a GET request via a non-persistent proxy HEAD and POST requests follow quite similarly in the usual case, just accounting for potential differences in Message Body. A basic test to perform a simple sanity check of your proxy is provided. Basic Persistent Proxy In this case, your proxy builds on the basic non-persistent proxy by allowing a client connection to be reused for multiple requests, which once again may include any combination of GET, HEAD, or POST requests. This is illustrated in the Figure 5 example of a typical persistent session. It mirrors the basic non-persistent proxy example, with the minor differences noted below. Important reminder: We do not expect your proxy to maintain persistent server connections. In step 2, when the proxy receives the request, it also records whether the client intends for the connection to persist based on the explicit inclusion or absence of a Connection: close header. In step 7, when the proxy sends the response to the client, the Connection header needs to be set according to the client preference. In step 8, assuming the client preference is for the connection to persist, the connection remains open for the proxy to receive a subsequent request. The request-response process is repeated, potentially any number of times, until step 14, when the connection is finally closed. This may occur for a number of reasons. For example, because the client has indicated their intention in the previous request to close the Connection, the client has timed out the proxy, or the proxy has timed out the client. 2025/7/18 17:54 Assignment Specification | COMP3331/9331 25T2 https://cgi.cse.unsw.edu.au/~cs3331/25T2/assign/index.html 12/25 Origin ServerClient Proxy
Request
Response
Request
Response
Open Connection
Close Connection
Open Connection
GET http://www.example.com/index.html HTTP/1.1\r\n Host: www.example.com\r\n Connection: keep-alive\r\n (more headers ...)\r\n \r\n GET /index.html HTTP/1.1\r\n Host: www.example.com\r\n Connection: close\r\n Via: 1.1 z1234567\r\n (more headers ...)\r\n \r\n HTTP/1.1 200 OK\r\n Content-Length: 1256\r\n Connection: close\r\n (more headers ...)\r\n \r\n
(more content ...) HTTP/1.1 200 OK\r\n Content-Length: 1256\r\n Connection: keep-alive\r\n Via: 1.1 z1234567\r\n (more headers ...)\r\n \r\n
(more content ...)
Request
Request
Open Connection
GET http://www.example.com/404.html HTTP/1.1\r\n Host: www.example.com\r\n Connection: keep-alive\r\n (more headers ...)\r\n \r\n GET /404.html HTTP/1.1\r\n Host: www.example.com\r\n Connection: close\r\n Via: 1.1 z1234567\r\n (more headers ...)\r\n \r\n 1 2 3 4 5 6 7 8 9 10 2025/7/18 17:54 Assignment Specification | COMP3331/9331 25T2 https://cgi.cse.unsw.edu.au/~cs3331/25T2/assign/index.html 13/25
Response
Response
Close Connection
Close Connection
HTTP/1.1 404 Not Found\r\n Content-Length: 1256\r\n Connection: close\r\n (more headers ...)\r\n \r\n
(more content ...) HTTP/1.1 404 Not Found\r\n Content-Length: 1256\r\n Connection: keep-alive\r\n Via: 1.1 z1234567\r\n (more headers ...)\r\n \r\n
(more content ...) // // 11 12 13 14 // Figure 5: A client sending multiple GET requests via a persistent proxy HTTP Tunneling with CONNECT At this point your proxy is quite functional, but it's limited to HTTP traffic, which is increasingly rare as most websites and services have transitioned to HTTPS for security and privacy. However, an HTTP proxy implementing the CONNECT method can handle HTTPS traffic by tunneling encrypted connections, allowing it to support modern web applications securely. Consider the typical example in Figure 6: 1. Client-Proxy Connection The client initiates a connection with the proxy, which the proxy accepts. 2. Client Request The proxy should start receiving and parsing the request message, as previously described, but in this case: Since the method is CONNECT, the request target is expected in authority-form. Since the method is CONNECT, no body is expected. The proxy does not forward this message and can ignore all headers, but the proxy does need to validate that the request target explicitly includes a port, and that the port is 443. 3. Proxy-Server Connection The proxy initiates a connection with the origin server specified in the request target, which the origin server accepts. Note: the proxy does not send any HTTP messages to the server. 4. Proxy Response The proxy sends an HTTP/1.1 200 Connection Established response to the client, with no additional headers and no body. 5. Raw Data Relay 2025/7/18 17:54 Assignment Specification | COMP3331/9331 25T2 https://cgi.cse.unsw.edu.au/~cs3331/25T2/assign/index.html 14/25 The proxy starts blindly forwarding data in both directions. Any data it receives on one socket, it writes to the other socket, until... 6. Connection Termination Some time later, when one of the endpoints closes its connection, the proxy ensures any outstanding data is written and gracefully closes both sockets. Origin ServerClient Proxy
Request
Response
Open Connection
Close Connection
Close Connection
Open Connection
CONNECT example.com:443 HTTP/1.1\r\n Host: www.example.com:443\r\n (more headers ...)\r\n \r\n HTTP/1.1 200 Connection Established\r\n \r\n 1 2 3 4 // // // Raw Data 5 6 Figure 6: A client sending a CONNECT request to a proxy When implementing support for the CONNECT method, the proxy must be able to blindly forward traffic in both directions without any awareness of message framing. This means that once the tunnel is established, the proxy simply relays raw data between the client and the server, without interpreting or modifying it. A na?ve single-threaded approach that reads from one socket and then writes to the other won t work reliably, as it may block indefinitely if one side isn't actively sending data. This can lead to deadlocks where neither endpoint is receiving data because the proxy is stuck waiting on a read operation. To avoid this, implementations typically use multi-threading, non-blocking I/O (e.g., select(), poll()), or asynchronous event loops to ensure data is forwarded as soon as it's available on either side. Error Handling Connections may close unexpectedly at any time, whether due to network conditions or intentional termination. Your proxy must handle such events gracefully to ensure robustness. Failure to do so may cause 2025/7/18 17:54 Assignment Specification | COMP3331/9331 25T2 https://cgi.cse.unsw.edu.au/~cs3331/25T2/assign/index.html 15/25 it to break under even basic test scenarios. Additionally, your proxy should handle explicit error conditions as outlined below. In most cases, it should generate and send an HTTP response with the appropriate status code and reason phrase. The response must include a representation (i.e., a response body) providing some minimal information about the error, including the mentioned required phrases. You may use any text-based format, such as text/html or text/plain, but ensure the response adheres to that format and includes properly set Content-Type and Content-Length headers. Explicit Error Conditions Handling Client Timeouts If a client is idle for timeout seconds, the proxy should gracefully close the socket and move on. Response: Not applicable Malformed CONNECT request If no port or a port other than 443 is specified: Response: 400 Bad Request with "invalid port" mentioned in the body. Invalid Request Target If the request target is missing a host: Response: 400 Bad Request with "no host" mentioned in the body. If the host/port in the request target points to the proxy itself (self-loop detection): Response: 421 Misdirected Request with "proxy address" mentioned in the body. Connection Issues If the origin server refuses the connection: Response: 502 Bad Gateway with "connection refused" mentioned in the body. If the origin server cannot be resolved via DNS: Response: 502 Bad Gateway with "could not resolve" mentioned in the body. If the origin server closes the connection unexpectedly: Response: 502 Bad Gateway with "closed unexpectedly" mentioned in the body. Note: A response is not required if this happens during HTTP tunneling. If the origin server fails to respond within timeout seconds: Response: 504 Gateway Timeout with "timed out" mentioned in the body. Note: A response is not required if this happens during HTTP tunneling. Caching Your proxy should implement an in-memory cache that stores cacheable responses. This is to reduce the response time and network bandwidth consumption on future equivalent requests. Parameters for the cache are provided as command-line arguments to the proxy, specifically: max_object_size in bytes, a strictly positive integer. max_cache_size in bytes, an integer at least equal to max_object_size. 2025/7/18 17:54 Assignment Specification | COMP3331/9331 25T2 https://cgi.cse.unsw.edu.au/~cs3331/25T2/assign/index.html 16/25 When considering the object or cache size, your proxy must only count bytes used to store the actual objects. Metadata, such as response headers and timestamps, will also need to be stored, but for the purposes of these calculations should be ignored. We take a very simplified view of HTTP caching, as outlined in this section, ignoring all cache directives that may be included in messages. A response is only cacheable if it is: received in response to a GET request; has a status code of 200; and the object is no greater than max_object_size. If caching such a response would cause the cache to exceed max_cache_size, then one or more responses must first be evicted using a least recently used (LRU) replacement policy. No more responses should be evicted than necessary. Upon receiving a GET request, your proxy should first check its cache to see if the request can be satisfied by a stored response. If it can, a "cache hit" occurs, and the proxy should respond directly using the stored response. Otherwise, a "cache miss" occurs, and the proxy should forward the request to the origin server. The normalised target URL of a GET request is the "cache key", which is used to identify a stored response. Normalisation of the target URL considers the following rules: No port is equivalent to the default port (80). No path is equivalent to "/". The scheme and host are case insensitive. All other components are case sensitive. For example, all of the following URLs are equivalent: http://example.com http://example.com/ HTTP://EXAMPLE.COM/ HTTP://EXAMPLE.COM:80/ HTTP://EXAMPLE.COM:0080/ While none of the following URLs are equivalent: http://example.com/ http://example.com/index.html http://example.com/INDEX.HTML http://example.com:8080/INDEX.HTML http://example.com:8080/INDEX.HTML?FOO=BAR Request methods other than GET should simply bypass the cache. If your proxy supports concurrency, then you should be particularly careful about how it interacts with any shared data structures. 2025/7/18 17:54 Assignment Specification | COMP3331/9331 25T2 https://cgi.cse.unsw.edu.au/~cs3331/25T2/assign/index.html 17/25 Note, marking will rely entirely on your logging to assess the operation of your cache. If your proxy does not produce a log, then it will not be possible to award any marks for caching. Logging Your proxy should write to standard output a log of each HTTP transaction as it's completed. The log should follow a variant of the Common Log Format, with the following syntax: host port cache date request status bytes Where: host: is the IP address of the client which made the request. port: is the port of the client which made the request. cache: is - for all requests other than GET, in which case it is either H for a cache hit or M for a cache miss. date: is the date, time, and time zone that the request was received, in strftime format %d/%b/%Y:%H:%M:%S %z, enclosed in square brackets. Please note that if your proxy supports concurrency, then transactions may appear to be logged out of order and is to be expected. request: is the request line from the client, as received by the proxy, enclosed in double quotation marks. status: is the HTTP status code returned to the client. bytes: is the size of the object returned to the client, not including the response headers. Here is an example: 127.0.0.1 56150 - [22/May/2025:10:17:24 +1000] "HEAD http://www.example.org/
HTTP/1.1" 200 0 127.0.0.1 56149 M [22/May/2025:10:17:24 +1000] "GET http://www.example.org/
HTTP/1.1" 200 648 127.0.0.1 56154 - [22/May/2025:10:17:24 +1000] "POST http://httpbin.org/post
HTTP/1.1" 200 480 127.0.0.1 56150 H [22/May/2025:10:17:25 +1000] "GET http://www.example.org/
HTTP/1.1" 200 648 127.0.0.1 56156 - [22/May/2025:10:17:25 +1000] "CONNECT api.github.com:443
HTTP/1.1" 200 0 127.0.0.1 56149 M [22/May/2025:10:17:24 +1000] "GET http://httpstat.us/404? sleep=5000 HTTP/1.1" 404 13 A basic test to perform a simple sanity check of your log format is provided. Concurrency Extend your proxy to handle multiple client connections concurrently. This means your proxy should be able to process requests from multiple clients at the same time, rather than handling them sequentially. You may 2025/7/18 17:54 Assignment Specification | COMP3331/9331 25T2 https://cgi.cse.unsw.edu.au/~cs3331/25T2/assign/index.html 18/25 use threading, multiprocessing, or asynchronous I/O to achieve this. Ensure proper synchronisation when accessing any shared resources, such as the cache, to prevent race conditions. Reminder: You must not use any ready-made server or HTTP libraries to implement any aspects of this assignment. Report Submit a short report (no more than 3 pages) named report.pdf. The report should cover the following sections: 1. Programming Language and Code Organisation State the programming language used (i.e., C, Java, or Python). Briefly describe your code structure (e.g., key directories, files like Makefile). 2. High-Level Design Provide a brief overview of your program s main components and their interactions. You may include a simple diagram if it helps clarify the design. 3. Data Structures Summarise the key data structures used in your proxy (e.g., for HTTP message handling, caching). Focus on the most important structures and how they support your design. 4. Limitations Mention any known limitations of your implementation (e.g., incomplete feature support, specific conditions where the program might fail). 5. Acknowledgments Indicate any external code or resources you ve used, with links or brief citations. Tips on Getting Started This is a complex assignment, and the best way to tackle a complex task is to start early and to do it in stages. Before attempting this asssignment, we advise that you read this specification and the FAQ in full, more than once, and finish Lab 2 and Lab 3. Lab 3, in particular, may serve as an excellent starting point for this assignment. 1. Understand Socket Basics Understand the difference between TCP (stream-oriented, reliable) and UDP (datagram-oriented, best-effort). Know the roles of client and server sockets in a TCP connection and the purpose of API calls such as: 2025/7/18 17:54 Assignment Specification | COMP3331/9331 25T2 https://cgi.cse.unsw.edu.au/~cs3331/25T2/assign/index.html 19/25 socket() C Creating a socket with appropriate domain (AF_INET for IPv4) and type (SOCK_STREAM for TCP). bind() C Binding a socket to an IP address and port (for servers). listen() C Marking a socket as passive to accept incoming connections. accept() C Accept a new client connection. connect() C Initiating a connection from the client to the server. TCP is stream-oriented data can be split across multiple recv() calls. The need for handling partial reads/writes, ensuring all expected data is sent and received. 2. Understand HTTP/1.1 Basics Make sure you have a good grasp of HTTP/1.1, particularly connection management, request/response structures, and important headers like Connection. Familiarise yourself with how HTTP proxies work within this framework. 3. Data Structure Design Spend some time thinking about your data structures. Designing good abstractions from the outset for things like HTTP messages and your cache will greatly simplify the proxy logic itself. 4. Plan and Document Draw a high-level architecture diagram to visualise how the components will interact. Organise your code with clear separation of concerns (e.g., separate modules for message parsing, caching, logging, etc.). Keep your code clean and document important design decisions. 5. Break Down the Problem Tackle the assignment in small steps, for example: First, focus on message parsing and basic proxy behavior for a single method like GET, before moving to HEAD and POST. Next, implement persistence, caching, and CONNECT. Finally, add concurrency handling once the foundation is solid. 6. Start with Logging Implement logging early to help with debugging and tracking the flow of requests and responses. Log all incoming requests, outgoing requests to the origin server, and the responses from the server. Keep the log format simple at first (e.g., log request details, response status, and any errors) so that you can easily trace issues during development, but eventially make sure logging conforms to the expected format. 7. Handle Errors Early and Gracefully Implement error handling from the beginning to prevent unexpected crashes. Handle timeouts, closed sockets, failed connections, and invalid HTTP requests gracefully. 8. Handle Basic Proxy Functionality 2025/7/18 17:54 Assignment Specification | COMP3331/9331 25T2 https://cgi.cse.unsw.edu.au/~cs3331/25T2/assign/index.html 20/25 Begin with the core proxy functionality: parsing HTTP requests, forwarding them to the origin server, and sending the server s response back to the client. This step will help you verify that the basic mechanics of the proxy are working before adding complexity. 9. Implement Client-Proxy Persistence After the basic proxy is functioning, handle the client-proxy connection persistence based on the client's Connection header. If the client requests a persistent connection, ensure that you respect this preference and keep the connection open as required. 10. Introduce Caching Implement a basic cache that stores responses from the origin server. Start by focusing on caching all responses and later introduce the notion of cacheable responses and the LRU replacement policy. Log the cache hits and misses so you can monitor whether your caching mechanism is working as expected. 11. Test and Debug Incrementally Regularly test your proxy with real HTTP requests (using tools like those suggested) to verify functionality and correctness. Use your logging to track issues and help identify where things go wrong, especially with caching or connection persistence. 12. Focus on Concurrency Later Once the basic functionality (logging, client-proxy persistence, caching) is working as expected, tackle concurrency. You can then start adding multithreading or an event-driven model to handle multiple client connections concurrently. Focus on ensuring correctness first concurrency can often add complexity that s easier to handle once your program works in a single-threaded fashion. 13. Read the Requirements Carefully Refer back to the assignment specifications regularly to ensure you're meeting all the requirements and haven t missed any crucial details. Testing and Debugging HTTP is a ubiquitous protocol, so fortunately there are many tools and services that you can use to debug and test your proxy. Simply bypassing your proxy also gives you a convenient mechanism to determine the expected response for a given request. Some tools and services are outlined in Useful Tools for Testing and Debugging. Marking will utilise similar tools. During development you should endeavour to use multiple user agents and communicate with as many origin servers as possible. 2025/7/18 17:54 Assignment Specification | COMP3331/9331 25T2 https://cgi.cse.unsw.edu.au/~cs3331/25T2/assign/index.html 21/25 It is imperative that you rigorously test your code to ensure that all possible (and logical) interactions can be correctly executed. Test, test, and test. Submission Please ensure that you use the mandated file names of report.pdf and, for the entry point of your application, one of: proxy.c proxy.py Proxy.java If you are using C or Java, then you must additionally submit a Makefile. This is because we need to know how to resolve any dependencies. See Sample Client-Server Programs and Networking Programming Resources for a guide on writing a Makefile. After running make, we should have one of the following executable files: proxy (for C) Proxy.class (for Java) Submission is via give using the following command syntax: $ give cs3331 assign
[ ... ] Note, this is the same command for both COMP3331 and COMP9331 students. If your codebase does not rely on a directory structure then you may submit the files directly. For example, assuming your implementation is in C, and you additionally have helper.c and helper.h files that your Makefile expects to find in the same directory as proxy.c: $ give cs3331 assign report.pdf Makefile proxy.c helper.c helper.h If your codebase relies on some directory structure, for example you've created helper functions or classes in a sub-directory to your main program, you must first tar the parent directory as assign.tar. For instance, assuming a directory assign contains all the relevant files and sub-directories (including your report), open a terminal and navigate to the parent directory, then execute: $ tar -cvf assign.tar assign $ give cs3331 assign assign.tar Please do not submit any build artefacts, test files/programs, or other particulars that are not required to compile and run your application. 2025/7/18 17:54 Assignment Specification | COMP3331/9331 25T2 https://cgi.cse.unsw.edu.au/~cs3331/25T2/assign/index.html 22/25 Upon running give, ensure that your submission is accepted. You may submit often. Only your last submission will be marked. Emailing your code to course staff will not be considered as a submission. Submitting the wrong files, failing to submit certain files, failing to complete the submission process, or simply failing to submit, will not be considered as grounds for re-assessment. If you wish to validate your submission, you may execute: $ 3331 classrun -check assign
# show submission status $ 3331 classrun -fetch assign
# fetch most recent submission Important: It is your responsibility to ensure that your submission is accepted, and that your submission is what you intend to have assessed. No exceptions. Late Submission Policy Late submissions will incur a 5% per day penalty, for up to 5 days, calculated on the achieved mark. Each day starts from the deadline and accrues every 24 hours. For example, an assignment otherwise assessed as 12/20, submitted 49 hours late, will incur a 3 day x 5% = 15% penalty, applied to 12, and be awarded 12 x 0.85 = 10.2/20. Submissions after 5 days from the deadline will not be accepted unless an extension has been granted, as detailed in Special Consideration and Equitable Learning Services. Special Consideration and Equitable Learning Services Applications for Special Consideration must be submitted to the university via the Special Consideration portal. Course staff do not accept or approve special consideration requests. Students who are registered with Equitable Learning Services must email cs3331@cse.unsw.edu.au to request any adjustments based on their Equitable Learning Plan. Any requested and approved extensions will defer late penalties and submission closure. For example, a student who has been approved for a 3 day extension, will not incur any late penalties until 3 days after the standard deadline, and will be able to submit up to 8 days after the standard deadline. Plagiarism Group submissions will not be allowed. Your programs must be entirely your own work. Plagiarism detection software will be used to compare all submissions pairwise (including submissions for similar assessments in previous years, if applicable) and serious penalties will be applied, including an entry on UNSW's plagiarism register. 2025/7/18 17:54 Assignment Specification | COMP3331/9331 25T2 https://cgi.cse.unsw.edu.au/~cs3331/25T2/assign/index.html 23/25 You are not permitted to use code generated with the help of automatic tools such as GitHub Copilot, ChatGPT, Google Bard. Do not copy ideas or code from others. Do not use a publicly accessible repository or allow anyone to see your code. Code generated by GitHub Copilot, ChatGPT, Google Bard and similar tools will be treated as plagiarism. Please refer to the online sources to help you understand what plagiarism is and how it is dealt with at UNSW: Plagiarism and Academic Integrity UNSW Plagiarism Policy UNSW Plagiarism Management Procedure Marking Rubric Important Reminder: Your proxy must compile and run within the CSE environment. Ensure it is thoroughly tested in that environment. Functionality Marks Basic Non-Persistent Proxy: ? - GET 2 ? - HEAD 1 ? - POST 1 Basic Persistent Proxy: ? - GET only 1 ? - GET + HEAD + POST 2 Via Header 1 CONNECT 2 Explicit Error Conditions 2 Logging 1 Caching 2 Non-Persistent Concurrency 1 Persistent Concurrency 1 Stress Test 1 Report 1 Code Quality 1 Total 20 2025/7/18 17:54 Assignment Specification | COMP3331/9331 25T2 https://cgi.cse.unsw.edu.au/~cs3331/25T2/assign/index.html 24/25 No particular coding style is mandated, just ensure your code style is consistent, your code is clean, and your code is
版权所有：编程辅导网 2021 All Rights Reserved 联系方式：QQ:99515681 微信：codinghelp 电子信箱：99515681@qq.com
免责声明：本站部分内容从网络整理而来，只供参考！如有版权问题可联系本站删除。
