Building a Basic HTTP Server with Python: A Guide for Automation and Prototyping

What is this article about

This article presents a basic example of a Python-based HTTP server. It can be useful for personal needs, whether it be automation or prototyping. Moreover, it can be of use to software test (automation) engineers who wish to mock external services for System Testing or to remove environment instability or dependency.

Thanks to Python and its libraries, starting or editing this server (which is less than 200 lines of code) is straightforward. All the code is available in this repo. Feel free to experiment and use it for your goals. However, I strongly recommend not using it with sensitive information due to the absence of cyber security checks and coverage.

The implementation

The Base HTTP server is a standard library in Python, provided since version 2.7. It offers classes for the server itself and the request handler.

To get started, we'll need to import the necessary libraries and start our server. It's important to maintain the do_GET and do_POST functions as they are. From the outset, we have:

the Server class, which aids in listening to a specific address and port,
the RequestHandler class, tasked with managing server requests via the do_GET and do_POST methods, which will be automatically invoked based on the HTTP method,
the start_server function that acts as a wrapper to initiate our server,
and finally, the main block that includes a standard ArgumentParser, offering basic help and facilitating the required server address and port input.

import argparse
from http.server import BaseHTTPRequestHandler, HTTPServer
from json import dumps


class Server(HTTPServer):
    def __init__(self, address, request_handler):
        super().__init__(address, request_handler)


class RequestHandler(BaseHTTPRequestHandler):
    def __init__(self, request, client_address, server_class):
        self.server_class = server_class
        super().__init__(request, client_address, server_class)

    def do_GET(self):
        response = {"message": "Hello world"}
        self.send_response(200)
        self.send_header("Content-type", "application/json")
        self.send_header("Content-Length", str(len(dumps(response))))
        self.end_headers()
        self.wfile.write(str(response).encode('utf8'))

    def do_POST(self):
        response = {"message": "Hello world"}
        self.send_response(200)
        self.send_header("Content-type", "application/json")
        self.send_header("Content-Length", str(len(dumps(response))))
        self.end_headers()
        self.wfile.write(str(response).encode('utf8'))


def start_server(addr, port, server_class=Server, handler_class=RequestHandler):
    server_address = (addr, port)
    http_server = server_class(server_address, handler_class)
    print(f"Starting server on {addr}:{port}")
    http_server.serve_forever()


def main():
    parser = argparse.ArgumentParser(description="Run a simple HTTP server.")
    parser.add_argument(
        "-l",
        "--listen",
        default="0.0.0.0",
        help="Specify the IP address which server should listen",
    )
    parser.add_argument(
        "-p",
        "--port",
        type=int,
        default=80,
        help="Specify the port which server should listen",
    )
    args = parser.parse_args()
    start_server(addr=args.listen, port=args.port)


if __name__ == "__main__":
    main()

With this setup, we already have a functional HTTP server that will respond to any GET and POST HTTP requests with the json:

{"message": "Hello world"}

Good, but let's make it more useful. To enhance this server, we're going to establish it as a classic "mock server". This requires incorporating various HTTP path handling (including links and methods), data storage, and fundamental server responses.

Next, I suggest we work with json and xml to give a demonstration. It's also generally useful to have a heartbeat method in place. It is a type of function that routinely checks system or network connections to ensure they're operating correctly.

First up, let's add a straightforward class to encapsulate the paths our server will use:

class Path:
    JSON = "/json"
    XML = "/xml"
    HEALTH = "/health"

Basic init data responses:

class Response:
    HEALTH_GOOD_RESPONSE = dumps({"message": "I'm doing great!"})
    DEFAULT_RESPONSE = dumps({"message": "Hello there. This is a default server response."})
    GOOD_RESPONSE = dumps({"message": "I got your data"})
    INIT_JSON_DATA = dumps({"status": True, "data": {"id": "FEFE", "some_field": "some_field_data"}})
    INIT_XML_DATA = '<?xml version="1.0" encoding="utf-8"?>' \
                    '<status>True</status>' \
                    '<data>' \
                    '<id>FEFE</id>' \
                    '<some_field>some_field_data</some_field>' \
                    '</data>'

And storage that will work only during runtime:

class Storage:
    def __init__(self):
        self.json = Response.INIT_JSON_DATA
        self.xml = Response.INIT_XML_DATA

    def write_json(self, data):
        self.json = data

    def read_json(self):
        return self.json

    def write_xml(self, data):
        self.xml = data

    def read_xml(self):
        return self.xml

We then introduce new classes during the Server and RequestHandler initialization to highlight dependency and support future modifications. Additionally, the RequestHandler will have access to this new data.

class Server(HTTPServer):
    def __init__(self, address, request_handler, storage, paths, response):
        super().__init__(address, request_handler)
        self.storage = storage
        self.path = paths
        self.response = response


class RequestHandler(BaseHTTPRequestHandler):
    def __init__(self, request, client_address, server_class):
        self.server_class = server_class
        self.response_json_data = server_class.storage.read_json()
        self.response_xml_data = server_class.storage.read_xml()
        super().__init__(request, client_address, server_class)

Now, we need to adjust our RequestHandler.doGET and RequestHandler.doPOST methods. The logic behind this is straightforward:

check the method type (BaseHTTPRequestHandler will handle this for us),
verify the path that was used in the request,
invoke the requested method or function.

def do_GET(self):
    if self.path == self.server_class.path.JSON:
        return self.json_response()

    if self.path == self.server_class.path.XML:
        return self.xml_response()

    if self.path == self.server_class.path.HEALTH:
        return self.return_health()

    return self.default_response()


def do_POST(self):
    if self.path == self.server_class.path.JSON:
        return self.store_json_test_data()

    if self.path == self.server_class.path.XML:
        return self.store_xml_test_data()

    return self.default_response()

In order to respond with a minimal HTTP data we need to provide:

Data type (specified in the header)
Data length (also in the header)
Response code
Response data You can find more information about HTTP headers here.

def set_json_headers(self, success_response=None):
    self.send_response(200)
    if success_response is not None:
        self.send_header("Content-type", "application/json")
    self.send_header("Content-Length", str(len(success_response)))
    self.end_headers()


def set_response(self, response):
    self.wfile.write(str(response).encode('utf8'))


def return_health(self):
    """
    Implementation for health method and response.
    """
    self.set_json_headers(self.server_class.response.HEALTH_GOOD_RESPONSE)
    self.set_response(self.server_class.response.HEALTH_GOOD_RESPONSE)

This method enables us to get a response on the /health path with the GET method, along with data.

{"message": "I'm doing great!"}

Next, we'll add the rest of the response methods. The logic will be exactly the same as for the /health method above.

def set_xml_headers(self, success_response=None):
    self.send_response(200)
    if success_response is not None:
        self.send_header("Content-type", "text/plain")
        self.send_header("Content-Length", str(len(success_response)))
    self.end_headers()


def default_response(self):
    """
    Implementation for default server response.
    """
    self.set_json_headers(self.server_class.response.DEFAULT_RESPONSE)
    self.set_response(self.server_class.response.DEFAULT_RESPONSE)


def json_response(self):
    """
    Response with json test data stored in runtime.
    """
    self.set_json_headers(self.response_json_data)
    self.set_response(self.response_json_data)


def xml_response(self):
    """
    Response with xml test data stored in runtime.
    """
    self.set_xml_headers(self.response_xml_data)
    self.set_response(self.response_xml_data)

Perhaps the "trickiest" part of this server is the store methods. We want these methods to receive data and store it in runtime, so it can be sent back upon request. This method gives a full response, and additionally, we need to save data to our class Storage variables. Here, it's necessary to parse data length from the Content-Length and read the data itself.

def store_json_test_data(self):
    """
    Store test data in runtime. Use it to upload a new one.
    """
    success_response = self.server_class.response.GOOD_RESPONSE
    self.set_json_headers(success_response)
    self.set_response(success_response)

    if self.headers.get('Content-Length') is not None:
        self.server_class.storage.write_json(self.rfile.read(int(self.headers['Content-Length'])).decode())


def store_xml_test_data(self):
    """
    Store test data in runtime. Use it to upload a new one.
    """
    success_response = self.server_class.response.GOOD_RESPONSE
    self.set_json_headers(success_response)
    self.set_response(success_response)

    if self.headers.get('Content-Length') is not None:
        self.server_class.storage.write_xml(self.rfile.read(int(self.headers['Content-Length'])).decode())

We are now good to go. Launch the server and use the following links to open it in your browser. Doing so will trigger the method execution. Try sending data to it and receiving it back:
http://0.0.0.0/health
http://0.0.0.0/some_path
http://0.0.0.0/json
http://0.0.0.0/xml

Next, let's enhance our default_response and "help" message with a handy feature. I'd like for it to display all available paths in its response data. To achieve this, we need to add a method to our Path class. Fortunately, the Inspect library will do all the heavy lifting for us.
This method examines all class attributes, filters them by "__" symbols, and returns our paths as a list.

@staticmethod
def get_paths():
    return [attr_pair[1] for attr_pair in
            inspect.getmembers(Path, lambda attr_name: not (inspect.isroutine(attr_name))) if
            not (attr_pair[0].startswith('__') and attr_pair[0].endswith('__'))]

Next, we add new code in the DEFAULT_RESPONSE definition to incorporate our new data into the output strings.

DEFAULT_RESPONSE = dumps({"message": "Hello there. This is a default server response. Try valid URLs: {0}".format(
        Path.get_paths())})

And ArgumentParser

parser = argparse.ArgumentParser(description="Run a simple HTTP server. List of available paths: {0}".format(
    Path.get_paths()))

Now we have useful information from the default server response. It guides users to the right paths.
Try http://0.0.0.0/some_path in your browser

{"message": "Hello there. This is a default server response. Try valid URLs: ['/health', '/json', '/xml']"}

At this point, our server is up and running, and easy to use. All the code can be found in the repo with some basic unit tests as a bonus. This should help you safely modify it for your needs.

Blog

Building a Basic HTTP Server with Python: A Guide for Automation and Prototyping

Artem Tregub

What is this article about

The implementation

Join Our Newsletter. No Spam, Only the good stuff.

Related