TheLazy.Dev
Portfolio GitHub

Blog

fastapi |python |chatgpt |api

Building and deploying a web API powered by ChatGPT

ruarfff

Ruairí O'Brien

Posted on May 24, 2023

Building and deploying a web API powered by ChatGPT

ChatGPT is an interesting tool. It makes certain things that used to be hard to do pretty easy. In some cases, it introduces new concepts you may not have thought to do before.

OpenAI provides some sdks to help build on top of ChatGPT.

In this post, we'll take a simple idea implemented using ChatGPT and deploy it. In a follow-up post, we'll look at how to operate the application in production. We will also look at how to overcome some of the challenges of using ChatGPT in a live application.

Please note: In these posts we end up signing up to various services. All but OpenAI have free plans. OpenAI has a free trial but does cost money when the free trial expires.

Our Sample App

In the DeeplearningAI Prompt Engineering Course, there is a section on summarising content.

This is an interesting function and one that would be quite tricky to do without using an LLM or some form of NLP.

In our sample application, we will use the NYTimes API to pull in the top stories for today. We will get ChatGPT to summarise the stories and we will show that summary.

The end result is available to view at: https://summer-ui.fly.dev/

The sample code for this is available in 2 repos:

Feel free to ignore the UI code if you wish as it's only there to display the data pulled from our API.

Tools

Our API is built using FastAPI which is a Python web framework.

We can use the NYTimes API to get some news and the OpenAI API to run ChatGPT.

We can deploy to fly.io.

For the UI we'll use vanilla JavaScript, CSS and HTML.

GitHub will make an appearance, with a section on building a deployment pipeline using GitHub actions. That will be in part 2.

We'll also end up using Python, Docker and Git.

Setup

Unfortunately, this full setup requires and lot of signing up for stuff and getting API Keys.

Image description

I hope you'll bear with me.

If you want a secure way to store these API keys locally, check out my post on that.

Get an NYTimes API Key

For this, follow the NYTimes setup guide.

In short:

  • Go here, click Create Account (unless you already have one).
  • Once logged in go to 'My Apps' and add a new app.
  • Once you select the new app, you should see an API key.

Copy that API key and make it available in an environment variable called NYTIMES_API_KEY.

Not sure what I mean by set an environment variable? Check out this post.

Get a ChatGPT API Key

Once you have your key, add it to an environment variable called OPENAI_API_KEY.

Note: OpenAI's API is a paid service. You will get a free trial when you sign up. If, like me, you signed up ages ago, forgot about it and have lost your free trial, you will need to add a payment method. Everything I did while testing this stuff cost about 5 cents but I added 5 euros to my account. Hopefully, you have the free trial.

Fly.io

If you want to make something available on the internet you need to deploy it somewhere. There are a vast number of options. Fly.io is one that is currently free. You can also setup a free database and even a free redis cache.

Sign up for a free account at https://fly.io/. That's all you need for now.

Creating the App

This is a step-by-step guide. You could skip this whole section, pull down the sample code and run it. If you would like a deeper explanation of all the bits though, read on.

FastAPI App

We're going to use Python for the backend API. FastAPI is a nice framework. Both OpenAI and NYTimes provide Python packages for interacting with their APIs.

This, again, is a step with many options. The following is one way to go about doing it.

Create a directory for your project. I called my app Summer because it sounds like summarise ¯\_(ツ)_/¯

Call yours whatever you wish but replace summer in the various examples.

I like to use a directory, under my home directory, called dev and put my repos in there.

Note: All these commands assume you're using a reasonable shell (not Cmd.exe).

mkdir ~/dev/summer-api
cd ~/dev/summer-api
Enter fullscreen mode Exit fullscreen mode

I'm using the -api suffix here because I want to keep my backend and frontent in two different repositories.

Initialize git so we can keep our changes in source control.

git init
Enter fullscreen mode Exit fullscreen mode

You must have Python installed on your system. I'm currently using Python 3.11 but most versions from 3.8 on should be fine. Consider using pyenv to manage your Python installations if you don't already.

Install poetry:

pip install poetry
Enter fullscreen mode Exit fullscreen mode

We'll use poetry as it simplifies dependency management and makes it easier to run virtual environments.

Setup our python project with poetry:

poetry init
Enter fullscreen mode Exit fullscreen mode

You will be asked for various inputs. Most of them are obvious. The main thing is to match the dependnecies.

Package name [summer-api]:
Version [0.1.0]:
Description []:  Summarising the news!
Author [Your name here, n to skip]:  Your name
License []:  MIT
Compatible Python versions [^3.11]:

Would you like to define your main dependencies interactively? (yes/no) [yes]
Enter fullscreen mode Exit fullscreen mode

You can add dependencies during the init or just skip that and paste the dependencies listed below into the pyproject.toml file resulting in this:

[tool.poetry]
name = "summer-api"
version = "0.1.0"
description = "Summarising the news!"
authors = ["Your name"]
license = "MIT"
readme = "README.md"
packages = [{include = "summer_api"}]

[tool.poetry.dependencies]
python = "^3.11"
fastapi = "^0.95.2"
uvicorn = {extras = ["standard"], version = "^0.22.0"}
pynytimes = "^0.10.0"
openai = "^0.27.7"
fastapi-cache2 = "^0.2.1"
redis = "^4.5.5"

[tool.poetry.group.dev.dependencies]
httpx = "^0.24.1"
pytest = "^7.3.1"
black = "^23.3.0"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
Enter fullscreen mode Exit fullscreen mode

Feel free to update the versions although they will not be guaranteed to work with the code in this post.

Make sure to run poetry install if you manually update the pyproject.yml file.

Create a directory for our source code.

mkdir app
Enter fullscreen mode Exit fullscreen mode

Create a main file.

touch app/__init__.py
touch app/main.py
Enter fullscreen mode Exit fullscreen mode

Open app/main.py in an editor and enter the following:

from fastapi import FastAPI

app = FastAPI()


@app.get("/")
def index():
    return {"msg": "Welcome to the News App"}
Enter fullscreen mode Exit fullscreen mode

Run the app:

poetry run uvicorn app.main:app --reload
Enter fullscreen mode Exit fullscreen mode

You should see output like this:

INFO:     Will watch for changes in these directories: ['~/dev/summer-api']
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [29115] using WatchFiles
INFO:     Started server process [29144]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
Enter fullscreen mode Exit fullscreen mode

You can go to http://localhost:8000/ in your browser or hit it with curl e.g.

curl http://localhost:8000
{"msg":"Welcome to the News App"}
Enter fullscreen mode Exit fullscreen mode

Get top stories from NYTimes

Create a client:

touch app/nytimes_client.py
Enter fullscreen mode Exit fullscreen mode

In that file, setup a call to get today's top stories:

import os

from pynytimes import NYTAPI
# This is the API Key we setup and added to the env earlier
API_KEY = os.getenv("NYTIMES_API_KEY", "")
nyt = NYTAPI(API_KEY, parse_dates=True)


def get_top_stories():
    return nyt.top_stories()
Enter fullscreen mode Exit fullscreen mode

To test it out, let's create and endpoint that simply returns the top stories.

In our app/main.py file import our new module:

from .nytimes_client import get_top_stories
Enter fullscreen mode Exit fullscreen mode

Add a new endpoint:

@app.get("/news")
def news():
    return get_top_stories()
Enter fullscreen mode Exit fullscreen mode

Run the app and test it (if the app was already running, the --reload flag will have made it restart so you don't have to run it again).

curl http://localhost:8000/news
Enter fullscreen mode Exit fullscreen mode

If you have jq installed you can use it to make the output look nicer.

curl http://localhost:8000/news | jq
Enter fullscreen mode Exit fullscreen mode

You should get a pretty big output here with a list of stories and lots of content.

Summarise the news

Now we'll introduce ChatGPT.

Create a new file:

touch app/summariser.py
Enter fullscreen mode Exit fullscreen mode

The code in this file is explained using inline comments.

import os

import openai

# You will need to get your API key from https://platform.openai.com/account/api-keys
openai.api_key = os.getenv("OPENAI_API_KEY")


# Got this function from this amazing course https://www.deeplearning.ai/short-courses/chatgpt-prompt-engineering-for-developers/
def get_completion(prompt, model="gpt-3.5-turbo"):
    messages = [{"role": "user", "content": prompt}]
    response = openai.ChatCompletion.create(
        model=model,
        messages=messages,
        temperature=0,  # this is the degree of randomness of the model's output
    )
    return response.choices[0].message["content"]


def summarise_news_stories(stories):
    """
    Takes in a list of news stories and prompts ChatGPT to
    generate a short summary of each story based on the provided
    Section, Subsection, Title, and Abstract.
    The function returns the summary generated by ChatGPT.

    Args:
    - stories (str): A list of news stories to be summarised.
    Each story should be a block of text with the following format
    where each part is on a newline:

    Section: the section
    Subsection: the subsection
    Title: the title
    Abstract: the Abstract

    The values for 'Subsection' and 'Abstract' can be empty
    strings if not applicable.

    Returns:
    - summary (str): A string containing the summary generated by ChatGPT
    for all the news stories.
    """

    print("Beginning summary")
    prompt = f"""
    Your task is to generate a short summary of a series of
    news stories given the Section, Subsection, Title and
    Abstract, of each story.

    The sections are after the 'Section:'.
    The subsections are after the 'Subsection:'. The subsections can be empty.

    The title is after the 'Title:'. The abstract is after the 'Abstract:'.
    The abstract can be empty.

    Summarise the stories below, delimited by triple backticks,
    in the style of an AI trying to convey information to humans.
    Please use paragraphs where appropriate and use at most 800 words.

    Stories: ```{stories}```
    """

    return get_completion(prompt)
Enter fullscreen mode Exit fullscreen mode

Formatting the stories

While we could probably have gotten ChatGPT to figure out the JSON from the NYTimes API, it's no harm to reduce the amount of stuff we send to it so let's format the stories to send to ChatGPT.

Create another file:

touch app/story_formatter.py
Enter fullscreen mode Exit fullscreen mode

Add a function to that file to format the stories so we only send the content we need:

def format_stories_to_string(stories):
    stories_string = ""
    for story in stories:
        title = story["title"]
        abstract = story["abstract"]
        section = story["section"]
        subsection = story["subsection"]
        stories_string += f"""
        Section: {section}
        Subsection: {subsection}
        Title: {title}
        Abstract: {abstract}
        """
    return stories_string
Enter fullscreen mode Exit fullscreen mode

The news endpoint

Now we can put everything together. Update our app/main.py file to look like the following:

import os

from fastapi import FastAPI, HTTPException

from .nytimes_client import get_top_stories
from .story_formatter import format_stories_to_string
from .summariser import summarise_news_stories

app = FastAPI()


@app.get("/")
def index():
    return {"msg": "Welcome to the News App"}


@app.get("/news")
def news():
    summary = ""
    images = []
    try:
        stories = get_top_stories()
        for story in stories:
            images.extend(story["multimedia"])
        summary = summarise_news_stories(format_stories_to_string(stories))
        images = list(
            filter(lambda image: image["format"] == "Large Thumbnail", images)
        )
    except Exception as e:
        print(e)
        raise HTTPException(
            status_code=500, detail="Apologies, something bad happened :("
        )
    return {"summary": summary, "images": images}
Enter fullscreen mode Exit fullscreen mode

You can now test it out but note, the call to ChatGPT takes a very long time. We will look at how to deal with that issue in part 2 of this series.

curl http://localhost:8000/news
Enter fullscreen mode Exit fullscreen mode

This should output a summary of the news as well as a list of all the related images.

There's another step we need before we setup a UI for this. Our API expects any calls in the browser to come from the same URL as our application. This is a security mechanism called Cross-Origin Resource Sharing (CORS). We need to update our API to allow requests from our UI.

To make that work, in our main.py add and configure the CORS middleware:

from fastapi.middleware.cors import CORSMiddleware

app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)
Enter fullscreen mode Exit fullscreen mode

Note: for allow_origins=["*"], you may want to be very specific once you know the URL your UI is at e.g. allow_origins=["http://localhost:8080"],. Using the wildcard here to reduce the chances of things going wrong while following these steps.

Setup our UI

This is totally optional so we will quickly go through the steps.

mkdir ~/dev/summer-ui
cd  ~/dev/summer-ui
git init
mkdir app
touch app/index.html
touch app/script.js
touch app/style.css
Enter fullscreen mode Exit fullscreen mode

In the app/index.html file:

<!DOCTYPE html>
<html>
<head>
    <title>Today's News</title>
    <link rel="stylesheet" type="text/css" href="styles.css">
    <meta http-equiv="Content-Security-Policy" content="upgrade-insecure-requests" />
</head>
<body>
    <main>
        <div id="loader">
            <h1>This should be quick, but ever so often it takes a long time...</h1>
            <div class="lds-circle"><div></div></div>
        </div>
        <div id="news-container" class="container hidden">
            <h1>Today's News</h1>
            <div class="date" id="date"></div>
            <p id="summary">
            </p>
        </div>
        <div id="images" class="img-container">
        </div>
    </main>
    <script src="script.js"></script>
</body>
</html>
Enter fullscreen mode Exit fullscreen mode

I am not good at CSS so I used ChatGPT to create some of this. I prompted it with:

Can you make some CSS that shows a bunch of images, defined in html img elements, and displays them like a bunch of physical photographs that have been spread out on a table?

It output some of the following which I have modified in places.

In the app/style.css file:

body {
  margin: 0;
  padding: 0;
  font-family: Arial, sans-serif;
  color: #333;
  line-height: 1.6;
  background: rgb(238, 174, 202);
  background: radial-gradient(
    circle,
    rgba(238, 174, 202, 1) 0%,
    rgba(148, 187, 233, 1) 100%
  );
}

.container {
  width: 80%;
  max-width: 800px;
  margin: 0 auto;
  padding: 20px;
  background-color: #fff;
  box-shadow: 0px 0px 20px rgba(0, 0, 0, 0.1);
  padding-bottom: 4rem;
}

h1 {
  font-size: 2em;
  margin-bottom: 0.5em;
  color: #212121;
  text-align: center;
}

p {
  font-size: 1em;
  text-align: justify;
}

/* Media queries for responsive design */
@media (max-width: 768px) {
  /* Tablets */
  .container {
    width: 90%;
  }

  h1 {
    font-size: 1.75em;
  }

  p {
    font-size: 0.9em;
  }
}

@media (max-width: 480px) {
  /* Phones */
  .container {
    width: 95%;
  }

  h1 {
    font-size: 1.5em;
  }

  p {
    font-size: 0.85em;
  }
}

.img-container {
  display: flex;
  flex-wrap: wrap;
  justify-content: center;
}

.photo {
  position: relative;
  margin: -15px; /* negative margin makes the images overlap */
  transform: rotate(-10deg); /* starting angle */
  box-shadow: 0 0 10px rgba(0, 0, 0, 0.5);
  transition: transform 0.5s;
  z-index: 1; /* ensure that images can stack on top of each other */
}

.photo:hover {
  transform: rotate(0deg); /* reset to straight when hovered */
}

.photo img {
  max-width: 150px;
  border-radius: 10px;
}

.date {
  text-transform: uppercase;
  text-align: center;
}

#loader {
  padding-top: 20%;
  text-align: center;
}

.lds-circle {
  display: inline-block;
  transform: translateZ(1px);
}
.lds-circle > div {
  display: inline-block;
  width: 64px;
  height: 64px;
  margin: 8px;
  border-radius: 50%;
  background: #fff;
  animation: lds-circle 2.4s cubic-bezier(0, 0.2, 0.8, 1) infinite;
}
@keyframes lds-circle {
  0%,
  100% {
    animation-timing-function: cubic-bezier(0.5, 0, 1, 0.5);
  }
  0% {
    transform: rotateY(0deg);
  }
  50% {
    transform: rotateY(1800deg);
    animation-timing-function: cubic-bezier(0, 0.5, 0.5, 1);
  }
  100% {
    transform: rotateY(3600deg);
  }
}

.hidden {
  display: none;
}
Enter fullscreen mode Exit fullscreen mode

In the app/script.js file:

window.onload = function () {
  fetch("https://summer-api.fly.dev/news/")
    .then((response) => response.json())
    .then((data) => {
      document.getElementById("loader").style.display = "none";
      document.getElementById("news-container").style.display = "block";

      const summaryElement = document.getElementById("summary");
      const imagesElement = document.getElementById("images");

      summaryElement.innerHTML = data.summary.split("\n").join("<br>");

      data.images.forEach((image, index) => {
        const img = document.createElement("img");
        img.className = "photo";
        img.src = image.url;
        img.alt = image.caption;
        img.style.width = "150px";
        img.style.height = "150px";

        if (index % 3 === 0) {
          img.style.gridColumnEnd = "span 2";
          img.style.gridRowEnd = "span 2";
        }

        imagesElement.appendChild(img);
      });
    })
    .catch((err) => console.error(err.message));

  function formatDate(date) {
    const options = {
      weekday: "long",
      month: "long",
      day: "numeric",
      year: "numeric",
    };
    return date.toLocaleDateString("en-US", options);
  }

  const today = new Date();
  document.getElementById("date").textContent = formatDate(today);
};
Enter fullscreen mode Exit fullscreen mode

Note: make sure to replace the URL in that script with whatever URL your API is at

Now you can serve that directory e.g. using the Live Server in VSCode or using some other utility e.g. serve.

All going well you'll see a page like this:

Image description

Deploying to fly.io

Let's start with the backend.

First, install flyctl https://fly.io/docs/hands-on/install-flyctl/

Login:

fly auth login
Enter fullscreen mode Exit fullscreen mode

Go to our API directory:

cd ~/dev/summer-api
Enter fullscreen mode Exit fullscreen mode

The easiest way to deploy a service is by using a Dockerfile to define the environment. If you can build and run it locally it'll probably work when deployed.

Create a Dockerfile.

touch Dockerfile
Enter fullscreen mode Exit fullscreen mode

With this content:

FROM python:3.11 as builder
WORKDIR /tmp

RUN pip install poetry

COPY ./pyproject.toml ./poetry.lock /tmp/

RUN poetry export -f requirements.txt --output requirements.txt --without-hashes

FROM python:3.11

WORKDIR /usr/app

COPY --from=builder /tmp/requirements.txt ./requirements.txt

RUN pip install --no-cache-dir --upgrade -r ./requirements.txt

COPY . .

EXPOSE 8080

CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8080"]
Enter fullscreen mode Exit fullscreen mode

Test it out:

docker build . -t summer-api
docker run -p 8080:8080 summer-api
Enter fullscreen mode Exit fullscreen mode

You should be able to hit your application at http://localhost:8080.

Fly knows how to deploy a Dockerfile.

fly launch
Enter fullscreen mode Exit fullscreen mode

That will guide you through setting up your app.

We also need to give it our API keys.

flyctl secrets set NYTIMES_API_KEY=${NYTIMES_API_KEY}
flyctl secrets set OPENAI_API_KEY=${OPENAI_API_KEY}
Enter fullscreen mode Exit fullscreen mode

All going well your API should be up an running. Mine for example is here: https://summer-api.fly.dev/

You will have a unique URL for your app.

For the UI we can do something similar.

cd ~/dev/summer-ui
Enter fullscreen mode Exit fullscreen mode

First, update the script in the UI so it points to the new URL for your API.

So this bit:

window.onload = function () {
  fetch("https://summer-api.fly.dev/news/")
Enter fullscreen mode Exit fullscreen mode

There, you should replace "https://summer-api.fly.dev/news/" with your URL.

Then create a Dockerfile:

touch Dockerfile
Enter fullscreen mode Exit fullscreen mode

Add the following content to the Dockerfile:

FROM pierrezemb/gostatic
COPY ./app/ /srv/http/

EXPOSE 8043
Enter fullscreen mode Exit fullscreen mode

8043 happens to be the port that image uses by default.

Run fly:

fly launch
Enter fullscreen mode Exit fullscreen mode

Fly launch creates a file in your repo called fly.toml.

I hit an issue where I needed to update that file to the following in my UI because it defaulted to the wrong port:

app = "summer-ui"
primary_region = "cdg"

[http_service]
  internal_port = 8043
  force_https = true
  auto_stop_machines = true
  auto_start_machines = true
  min_machines_running = 0
Enter fullscreen mode Exit fullscreen mode

You may or may not need this. Make sure to update the app and primary_region to the correct values for your app.

Improvements

This post is already huge so I will put all the improvements into their own post.

In the next post we'll look at:

  • setting up a continuous delivery pipeline
  • use redis to cache ChatGPT responses and speed up our site

Thank you for reading!

💖 💪 🙅 🚩
ruarfff
Ruairí O'Brien

Posted on May 24, 2023

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related

Building and deploying a web API powered by ChatGPT
fastapi Building and deploying a web API powered by ChatGPT

May 24, 2023