Previously we made a neat little command line tool that we could use to help us build a better resume. Now we will be deploying that application to the cloud!

I will be using the AWS CDK because I love it and it makes things like this relatively easy. You could certainly adapt these instructions to Terraform if you wanted to. First lets get a CDK project going and then we can talk about what we will need to provision for this part of the project.

Start a new CDK project

Create a new project directory and cdk init the project.

mkdir resume-assistant
cd resume-assistant
cdk init app --language python

Set up a virtual environment and install packages.

python3 -m venv .venv
source .venv/bin/activate   
pip install -r requirements.txt

Now that we have our repo set up, here are the things we are going to be putting in it.

1. A directory with our Lambda code (we will be modifying it a bit)
2. CDK code to create an S3 bucket to store our conversation "state"
3. CDK code to create a Lambda Function to run our code
4. CDK code to create a function URL so that we can call the function

Lets get started!

A new place for your python application to live

Create a new folder at the root of your CDK repo called chat_lambda and make a copy of your app.py and requirements.txt files from the chat_app directory from our previous session there.

We are essentially forking that code and will be modifying each of them in different ways to get to where we want to go. This chat_lambda directory is where the CDK is going to be told the code for a lambda function lives. Assuming you have docker running (make sure you have docker running) the CDK will create a container based on the runtime you specify, install the dependencies in requirements.txt, add your code, and then zip the whole thing up and deploy it as a Lambda function. Its pretty cool. Let's start Lambda-fying our code.

First, for some recent reason we need to decrease

Tradition dictates that we enter into the function at a lambda_handler method that takes some parameters (specifically an event parameter). Lets refactor with the lambda_handler and expect that our request that we are going to be sending is going to be in the body of the event.

# chat_lambda/app.py

import openai
import os
import json
import logging

openai.api_key = os.environ.get('OPENAI_API_KEY')

logger = logging.getLogger()
logger.setLevel(logging.INFO)


def lambda_handler(event, context):
    logger.info(event)

    payload = json.loads(event["body"])
    resume = payload["resume"]
    jd = payload["jd"]

    messages = [
        {"role": "system", "content": "You are a resume review assistant. You will be provided information about a job candidate. that information might be in the form of a formatted resume, or just a sentence about themselves. You also may also receive a description of a job or position sought by the candidate. Your task as the assistant is to prompt the candidate for additional information they could add to their resume to make it better generally, and more well suited for the job they are seeking specifically. Don't shy away from asking and promoting transferrable and soft skills."},
    ]
    messages.append({"role": "user", "content": f"Candidate information: {resume}"})
    messages.append({"role": "user", "content": f"Description of the job they want: {jd}"})

    completion = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=messages,
    )

    assistant_response = completion.choices[0].message.content
    logger.info(assistant_response)

    return {
        "statusCode": 200,
        "headers": {
            "Access-Control-Allow-Headers": "*"
        },
        "body": json.dumps({
            "message": assistant_response
            })
    }

So you will note that we also added some logging. Because it is at the INFO level we will not be able to see it in stdout if we were running this locally, but it WILL log in Lambda and be available in CloudWatch to help us troubleshoot if we did something wrong. And we usually do!

You will also note that the function will be looking for the resume and job description as the values of resume and jd in the body of the event. When we call this function we will need to make sure we supply them.

Finally, we are returning the assistant's response in a way that will not really allow us to continue this conversation. That will come later.

S3 Bucket for state

The bucket is simple enough. We will ensure that the objects are encrypted and non-public, and set an index. Because we are just playing around, we will also set the removal_policy and auto_delete_objects to the values that make cleanup easier. We will make this before the function itself so that we can pass in its identity to the function as an environmental variable.

# resume_assistant/resume_assistant_stack.py

from aws_cdk import aws_s3 as s3
from aws_cdk import RemovalPolicy

class ResumeAssistantStack(Stack):
    def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
        super().__init__(scope, construct_id, **kwargs)

        chat_log_bucket = s3.Bucket(self, "ChatLogBucket",
            versioned=True,
            encryption=s3.BucketEncryption.S3_MANAGED,
            enforce_ssl=True,
            block_public_access=s3.BlockPublicAccess.BLOCK_ALL,
            removal_policy=RemovalPolicy.DESTROY,
            auto_delete_objects=True,
            public_read_access=False,
        )

Note: all future additions to the CDK code (minus module imports) will be done inside that __init__ function in the ResumeAssistantStack class at the same indentation level as chat_log_bucket above. So that means if you copy and paste code blocks you will have to do some indentation yourself.

Lambda Function

The star of this application! We will need to refactor a number of things in the body of the code of the function ( the one that we "forked" before), but first, lets create the function infrastructure in the CDK. To get the CDK to actually build and package your lambda, you need to use an alpha module.

# requirements.txt

aws-cdk.aws-lambda-python-alpha

Install it with

pip install -r requirements.txt

Now you will be able to use both modules in your CDK code.
Lets also add permission for your lambda to read and write to that last bucket.

# resume_assistant/resume_assistant_stack.py

from aws_cdk import aws_lambda_python_alpha as p_lambda
from aws_cdk import aws_lambda as _lambda
from aws_cdk import Duration
import os

chat_lambda = p_lambda.PythonFunction(self, "ChatFunction",
    runtime=_lambda.Runtime.PYTHON_3_10,
    timeout=Duration.seconds(60),
    memory_size=256,
    entry="chat_lambda",
    index="app.py",
    handler="lambda_handler",
    environment={
        "CHAT_LOG_BUCKET": chat_log_bucket.bucket_name,
        "OPENAI_API_KEY": os.environ.get("OPENAI_API_KEY", "empty")
    },
)
chat_log_bucket.grant_read_write(chat_lambda)

There are a number of things to note in the above code. First, the lambda doesn't have much memory provisioned for it. That is probably okay. It really isn't going to be doing much computing, just sending requests to OpenAI and then passing on the response. Second, we are setting OPENAI_API_KEY from our local environmental variables because I'm going to be building and deploying this locally and therefore it will have access to my environmental variables. If you were going to build and deploy this from a pipeline or something, you would have to make sure that your pipeline was also loading that value from a pipeline secret.

This is your second reminder to PLEASE remember to set really low usage limits for yourself in the OpenAI billing tab. You likely won't ever spend more than a dollar or so, but if you make this public, then you don't want a billing surprise.

Function URL

Because the time it is going to take for the OpenAI API to respond to us in a lot of cases (eg. when we tell it to use all of the context we have given it to parse a fully formatted resume) we can't use API Gateway which times out after 29 seconds. We are going to instead use Lambda Function URLs. We will leave CORS open for now, and be sure to make an output of that function URL.

# resume_assistant/resume_assistant_stack.py

from aws_cdk import aws_lambda as _lambda
from aws_cdk import CfnOutput

function_url = chat_lambda.add_function_url(
    auth_type=_lambda.FunctionUrlAuthType.NONE,
    cors=_lambda.FunctionUrlCorsOptions(
        allowed_origins=["*"],
        allowed_headers=["*"],
    )
)

CfnOutput(self, "ChatFunctionURL", value=function_url.url)

Now, your S3 bucket, function and function URL should be deployed to AWS and you should have the address of your function URL.

Lets Start Testing

We are now going to test our function and function URL. Lets go back to the python project that we had from before we copied it into the CDK project and modify it so that it calls our Lambda URL instead of the OpenAI API.

We are going to start simple and set those resume and jd values as the files we have, send them to the Lambda via the Lambda URL and see if we get a coherent response.

# chat_app/app.py

import requests

# Add your function URL from your CDK output here
function_url = "https://th1si5n0tmyactua1lam8daur1.lambda-url.us-east-1.on.aws/"

with open("resume.txt") as f:
    resume = f.readlines()

with open("jd.txt") as f:
    jd = f.readlines()

response = requests.get(function_url, json={ # this will be added to the event body
    "resume": resume,
    "jd": jd
})

print(response.json()["message"])

And my response:

⚡  python app.py
Can you please provide me with more details on your responsibilities as a team member at Old Navy? Also, what were some of your duties as a crew member at McDonald's? This would help me understand your previous roles and identify the transferable skills you have that align with the requirements mentioned in the job description.

Nice! So we are now sending that initial request to our Lambda, it is adding our system message and sending that initial completion request to OpenAI, parsing the response and sending it back to us!

But we need to continue a conversation

Yes, we will need to refactor a little bit in order to hold a conversation. We will need a way of storing that conversation (the S3 bucket) and a way to identify which conversation there is the one we are having now.

Here is the plan for that. We are going to refactor our Lambda to look for some things in addition to the jd and the resume. We are going to add to the body of our request some additional information: a uuid and a user_response.

The uuid key will hold an empty string "" as its value when we initiate the conversation and this will tell the Lambda that it is a new conversation and to generate a new uuid to track the conversation. This uuid will then be returned to the user in the response so that the user can pass it back to the Lambda with its next request allowing it to continue the conversation.

This will also allow us to store (and retrieve) the state of the conversation by putting (and getting) an object named <uuid>.txt in the S3 bucket we made.

The user_response key will initially also hold an empty string as its value at first (because the first message will just contain the resume and jd), but will hold the replies of the user as the conversation continues.

Lets first modify the requirements of the Lambda code so that it can use AWS's boto3 python library.

# chat_lambda/requirements.txt

boto3>=1.26

Now we can use boto to work with the S3 bucket in our Lambda.

# chat_lambda/app.py

import openai
import os
import json
import logging
import boto3
import uuid

openai.api_key = os.environ.get("OPENAI_API_KEY")
CHAT_LOG_BUCKET = os.environ.get("CHAT_LOG_BUCKET")
s3 = boto3.client('s3',
    region_name=os.environ.get('AWS_REGION', "us-east-1"))


logger = logging.getLogger()
logger.setLevel(logging.INFO)

def load_file_from_s3(bucket, key):
    response = s3.get_object(
        Bucket=bucket,
        Key=key,
    )
    return response['Body'].read().decode('utf-8')

def save_file_to_s3(bucket, key, data):
    response = s3.put_object(
        Bucket=bucket, 
        Key=key, 
        Body=data,
    )
    return response

def lambda_handler(event, context):
    logger.info(event)

    payload = json.loads(event["body"])
    chat_uuid = payload["uuid"]
    resume = payload["resume"]
    jd = payload["jd"]
    user_response = payload["user_response"]

    if chat_uuid == "": # this is a new chat
        chat_uuid = str(uuid.uuid4())
        logger.info(f"Creating new uuid: {chat_uuid}")
        messages = [
            {"role": "system", "content": "You are a resume review assistant. You will be provided information about a job candidate. that information might be in the form of a formatted resume, or just a sentence about themselves. You also may also receive a description of a job or position sought by the candidate. Your task as the assistant is to prompt the candidate for additional information they could add to their resume to make it better generally, and more well suited for the job they are seeking specifically. Don't shy away from asking and promoting transferrable and soft skills."},
        ]
        messages.append({"role": "user", "content": f"Candidate information: {resume}"})
        messages.append({"role": "user", "content": f"Description of the job they want: {jd}"})
        logger.info(save_file_to_s3(CHAT_LOG_BUCKET, f"chat_history/{chat_uuid}.txt", json.dumps(messages)))

    else: # this is an existing chat
        logger.info(f"Loading uuid: {chat_uuid}")
        messages = json.loads(load_file_from_s3(CHAT_LOG_BUCKET, f"chat_history/{chat_uuid}.txt"))
        messages.append({"role": "user", "content": user_response})


    completion = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=messages,
    )

    assistant_response = completion.choices[0].message.content
    messages.append({"role": "assistant", "content": assistant_response})
    logger.info(save_file_to_s3(CHAT_LOG_BUCKET, f"chat_history/{chat_uuid}.txt", json.dumps(messages)))

    return {
        "statusCode": 200,
        "headers": {
            "Access-Control-Allow-Headers": "*"
        },
        "body": json.dumps({
            "message": assistant_response,
            "uuid": chat_uuid,
            })
    }

You should primarily note the addition of the save and load to S3 methods and how they make use of uuid value and how the function flows depending on if it is a new conversation without a uuid or a conversation follow-up that arrives with a pre-existing uuid.

Now lets modify our python application that calls the Lambda to pass in the required information and loop over the request to our Lambda until we exit out. Note I purge the resume and jd values after the initial submission. We don't need to be flinging around all that extra data.

# chat_app/app.py

# Add your function URL from your CDK output here
function_url = "https://th1si5n0tmyactua1lam8daur1.lambda-url.us-east-1.on.aws/"

with open("resume.txt") as f:
    resume = f.readlines()
with open("jd.txt") as f:
    jd = f.readlines()
user_response = ""
chat_uuid = ""
while user_response not in ["quit", "Quit", "stop", "Stop", "exit", "Exit"]:
    response = requests.get(function_url, json={
        "resume": resume,
        "jd": jd,
        "uuid": chat_uuid,
        "user_response": user_response
    })
    print(response.json()["message"])
    chat_uuid = response.json()["uuid"]
    user_response = input("\nResponse:\n")
    resume = ""
    jd = ""

When we run this we will see that we can continue our conversation. Users are not exposed to any of the interesting stuff that you may have going on in the Lambda like your system message. Yay!

So now that we have the back-end of this application working in the cloud, our final post will be setting up some sort public static site that we can let users use instead of forcing them to use the command line! Woop woop!