Building a PDF Generator on AWS Lambda with Python3 and wkhtmltopdf

Update 07/18/2019: Source Code: https://github.com/rbk/Lambda-PDF-Generator

UPDATE 06/28/2019: For anyone attempting to follow this please read a follow up post about a font bug on the generated PDFs: https://blog.richardkeller.net/wkhtmltopdf-bad-kerning-on-aws-lambda/

Introduction

Creating scalable APIs in 2019 is easier than ever before with serverless auto-scaling compute power being widely accessible. In this article I'll show you how I created a PDF generator API that can handle with 1000 concurrent requests.

The purpose of this artice is to show you how I accomplished creating the API so that you will see how easy it is to create your own serverless APIs. When this project is complete, you will have an API endpoint in which you can POST a JSON object to, and recieve a link to a PDF on Amazon S3. The JSON object looks like this:

{
  "filename": "sample.pdf",
  "html": "<html><head></head><body><h1>It works! This is the default PDF.</h1></body></html>"
}

Setup

What you'll need:

Python3
Aws CLI installed and configured
Serverless installed

Serverless

The first thing you need to do is initialize a new Serverless project. Serverless is a tool that greatly simplifies using AWS, Gcloud, and Azure services so you can create APIs without worrying about managing a server. If you need more information about Serverless visite their website: Serverless.com. In your terminal run the following command to initialize your project.

Note: sls is short for serverless and can be used interchangably.

sls create --template aws-python3

This command will bootstrap a Python3 Lambda setup for you to work from.

WKHTMLTOPDF Binary

Next we need to get the binary for turning HTML into PDFs. For this I used WKHTMLTOPDF. Important: You have to use version 0.12.4 becaused later versions of wkhtmltopdf require dynamic dependencies of the host system that cannot be installed on Lambda.

Download version 0.12.4 here:

https://github.com/wkhtmltopdf/wkhtmltopdf/releases/download/0.12.4/wkhtmltox-0.12.4_linux-generic-amd64.tar.xz

Once you have extract this tar file, copy the binary wkhtmltopdf to the binary folder in your project.

./binary/wkhtmltopdf

More information about wkhtmltopdf can be found on their website:
WKHTMLTOPDF

Python3 Dependencies

Now that we have the WKHTMLtoPDF binary, we need the Python library, pdfkit to use it. Since we are using Serverless and AWS Lambda we cannot just run pip install pdfkit. We need a Serverless plugin to install our dependencies on Lambda.

In our project folder install the python plugin requirements module for Serverless.

sls plugin install -n serverless-python-requirements

Now, in your serverless.yml file, you need to add a custom section in the yaml:

custom:
  pythonRequirements:
    dockerizePip: true

Once the serverless plugin requirements is installed, you can add a requirements.txt file to your project and it will be automatically installed on lambda when you deploy.

Your requirements.txt for this project only needs to have pdfkit.

Requirements.txt

pdfkit

For any issues with this module checkout the repository issue:
Serverless Python Requirements

Ready, Set, Code

We are ready to code now. We only need to work with two files for this project: serverless.yml ad handler.py. Typically you want your serverless projects to be lightweight and independent of other parts of your codebase.

Serverless.yml

The serverless.yml file is our main configuration file. A serverless.yml file is broken down into a few important sections. If you are new to YAML, checkout the documentation for YAML.

For this serverless.yml configuration you'll need to create a file called config.yml. This will store the S3 bucket name. The serverless.yml will reference the config.yml to setup the correct bucket for your project.

Contents of config.yml

BucketName: 'your-s3-bucket-name'

Here is a high level overview of a serverless.yml file:

service: pdf-services # name or reference to our project
provider: # It is possible to use Azure, GCloud, or AWS
functions: # Array of functions to deploy as Lambdas
resources: # S3 buckets, DynamoDB tables, and other possible resources to create
plugins: # Plugins for Serverless
custom: # Custom variables used by you or plugins during setup and deployment

Our serverless configuration will do a few things for us when we deploy:

Create an S3 bucket called pdf-service-bucket to store our PDFs
Create a function that will create the PDFs
Give our function access to the S3 bucket
Setup an API endpoint for our Lambda function at:

POST https://xxxx.execute-api.xxxx.amazonaws.com/dev/new-pdf

Here is the full serverless.yml configuration. I've added a couple important comments in the code.

service: pdf-service
provider:
  name: aws
  runtime: python3.7
  # Set environment variable for the S3 bucket
  environment:
    S3_BUCKET_NAME: ${file(./config.yml):BucketName}
  # Gives our functions full read and write access to the S3 Bucket
  iamRoleStatements:
    -  Effect: "Allow"
       Action:
         - "s3:*"
       Resource:
          - arn:aws:s3:::${file(./config.yml):BucketName}
          - arn:aws:s3:::${file(./config.yml):BucketName}/*
functions:
  generate_pdf:
    handler: handler.generate_pdf
    events:
      - http:
          path: new-pdf
          method: post
          cors: true
resources:
 # Creates an S3 bucket in our AWS account
 Resources:
   NewResource:
     Type: AWS::S3::Bucket
     Properties:
       BucketName: ${file(./config.yml):BucketName}
custom:
  pythonRequirements:
    dockerizePip: true
plugins:
  - serverless-python-requirements

Handler.py

Handler.py is the only Python file we need. It contains one function to generate the PDF and save it to Lambda. You can creat more functions in this file to split the code into reusable parts, but for this example one function was enough. Lambda functions take two arguments by default: Event and Context.

Context contains environment variables and system information.

Event contains request data that is sent to the lambda function.

In this project we will send our function generate_pdf a filename and HTML, and it will return the URI for a PDF it creates.

import json
import pdfkit
import boto3
import os
client = boto3.client('s3')

# Get the bucket name environment variables to use in our code
S3_BUCKET_NAME = os.environ.get('S3_BUCKET_NAME')

def generate_pdf(event, context):

    # Defaults
    key = 'deafult-filename.pdf'
    html = "<html><head></head><body><h1>It works! This is the default PDF.</h1></body></html>"

    # Decode json and set values for our pdf    
    if 'body' in event:
        data = json.loads(event['body'])
        key = data['filename']
        html = data['html'] 

    # Set file path to save pdf on lambda first (temporary storage)
    filepath = '/tmp/{key}'.format(key=key)

    # Create PDF
    config = pdfkit.configuration(wkhtmltopdf="binary/wkhtmltopdf")
    pdfkit.from_string(html, filepath, configuration=config, options={})


    # Upload to S3 Bucket
    r = client.put_object(
        ACL='public-read',
        Body=open(filepath, 'rb'),
        ContentType='application/pdf',
        Bucket=S3_BUCKET_NAME,
        Key=key
    )

    # Format the PDF URI
    object_url = "https://{0}.s3.amazonaws.com/{1}".format(S3_BUCKET_NAME, key)

    # Response with result
    response = {
        "headers": {
            "Access-Control-Allow-Origin": "*",
            "Access-Control-Allow-Credentials": True,
        },
        "statusCode": 200,
        "body": object_url
    }

    return response

Deploy

Now that you have all the code ready it is time to deploy. In your project folder run the following command from your terminal:

sls deploy

After you run deploy, Serverless will create everything for you in AWS. You will get HTTP POST endpoint that you will use to generate PDFs. The endpoint will look something like this:

https://xxxxxx.execute-api.us-east-1.amazonaws.com/dev/new-pdf

You can use curl to test your function. The following curl command posts a JSON object to the lambda endpoint. The JSON object contains a filename and some html to turn into a PDF.

curl -d '{"filename":"my-sample-filename.pdf", "html":"<html><head></head><body><h1>Custom HTML -> Posted From CURL as {JSON}</h1></body></html>"}' -H "Content-Type: application/json" -X POST REPLACE-WITH-YOUR-ENDPOINT

Note: Replace "REPLACE-WITH-YOUR-ENDPOINT" with the endpoint you receive from Serverless.

After running this command you should receive the URI to your generated PDF.

Conclusion

Creating a scalable PDF generator is easy with of AWS and Serverless. In addition to having a scalable API, you don't have to worry about server maintenance. I encourage you to create more projects with AWS Lambda to get comfortable with the platform and configuration.

Thanks for reading!