TheLazy.Dev
Portfolio GitHub

Blog

python |mongodb |kubernetes |fastapi

Adding A Database to Your Application Using Okteto Stacks

kvng_zeez

Abdulazeez Abdulazeez

Posted on April 20, 2021

Adding A Database to Your Application Using Okteto Stacks

Temporary in-app databases are not ideal for web applications. Any unfortunate event such as an abrupt shutdown or restarting of the application will lead to the total loss of stored data.

This is the third post in our series on how to develop a FastAPI application with Okteto. In the previous posts, you learned how to deploy applications directly from your console using Okteto Stacks and how to deploy it directly from Okteto's UI.

In this tutorial, you will be adding a database to store better your application data, and then you will deploy the updated version to your Okteto namespace directly from your command line.

Initial Setup

Start by creating a fork of the application from GitHub and then clone it locally.

git clone https://github.com/okteto/fastapi-crud
Enter fullscreen mode Exit fullscreen mode

In your local clone, set it up using the following commands:

$ cd fastapi-crud
$ python3 -m venv venv && source venv/bin/activate
(venv)$ pip install -r requirements.txt
Enter fullscreen mode Exit fullscreen mode

Verify that the setup is complete by running:

python3 main.py
Enter fullscreen mode Exit fullscreen mode

After verifying the installation, update the DOCKERHUBUSERNAME variable in your stack file to okteto.dev. Deploy the application with the command:

(venv)$ okteto stack deploy --build
Enter fullscreen mode Exit fullscreen mode

Creating A New Branch

For this article, we will create a new branch. This is to enable you to differentiate the first post in the series from this.

From your console, run the command:

git checkout -b mongo-crud
Enter fullscreen mode Exit fullscreen mode

The command above creates a new branch mongo-crud with the origin pointing to main.

Adding A Database

Before proceeding, verify that you have MongoDB installed or proceed to the MongoDB's Installation page to install MongoDB.

To avoid data loss, you're going to rewrite the application logic to use a MongoDB database. Later in the post you'll deploy the database in your Okteto namespace, but first, let's run it locally to verify that the code works perfectly.

Start by installing pymongo, a MongoDB driver for Python application, python-decouple for reading environment secrets and update the requirements.txt file:

(venv)$ pip install pymongo
Enter fullscreen mode Exit fullscreen mode

Update your requirements.txt file:

...
pymongo
python-decouple
Enter fullscreen mode Exit fullscreen mode

In the api folder, create a new file, database.py, where you'll write the database CRUD operations' functions.

Start by importing MongoClient, ObjectID and config:

from pymongo import MongoClient
from bson import ObjectId
from decouple import config
Enter fullscreen mode Exit fullscreen mode

MongoClient is responsible for the connection from our application to the database, ObjectId, on the other hand, is used to pass id values in MongoDB properly, and config is responsible for reading application secrets from .env files.

Next, define the connection, database and database collection details:

connection_details = config("DB_HOST")

client = MongoClient(connection_details)

database = client.recipes

recipe_collection = database.get_collection('recipes_collection')
Enter fullscreen mode Exit fullscreen mode

On the first line above, you are using the decouple library to read the environment variable DB_HOST. Create a .env file in the root folder containing the connection detail:

DB_HOST=mongodb://localhost:27017
Enter fullscreen mode Exit fullscreen mode

Documents in MongoDB are stored in JSON format and the _id in ObjectId format. Write a function to parse the result from a query:

def parse_recipe_data(recipe) -> dict:
    return {
        "id": str(recipe["_id"]),
        "name": recipe["name"],
        "ingredients": recipe["ingredients"]
    }
Enter fullscreen mode Exit fullscreen mode

CRUD functions

The next step is to write the functions responsible for saving, removing, updating and deleting recipes. Start by implementing the save_recipe:

def save_recipe(recipe_data: dict) -> dict:
    recipe = recipe_collection.insert_one(recipe_data).inserted_id
    return {
        "id": str(recipe)
    }
Enter fullscreen mode Exit fullscreen mode

The function above inserts the recipe data into the database and returns the newly created recipe's ID.

Next, the function for retrieving a single recipe and all the recipes from the database:

def get_single_recipe(id: str) -> dict:
    recipe = recipe_collection.find_one({"_id": ObjectId(id)})
    if recipe:
        return parse_recipe_data(recipe)

def get_all_recipes() -> list:
    recipes = []
    for recipe in recipe_collection.find():
        recipes.append(parse_recipe_data(recipe))

    return recipes
Enter fullscreen mode Exit fullscreen mode

The first function above returns a single recipe whose ID matches the supplied one and an error message if it doesn't exist, while the second function returns all the contained recipes in the database.

Next, write the update_recipe_data function responsible for updating recipe data:

def update_recipe_data(id: str, data: dict):
    recipe = recipe_collection.find_one({"_id": ObjectId(id)})
    if recipe:
        recipe_collection.update_one({"_id": ObjectId(id)}, {"$set": data})
        return True
Enter fullscreen mode Exit fullscreen mode

Lastly, write the function for deleting a recipe:

def remove_recipe(id: str):
    recipe = recipe_collection.find_one({"_id": ObjectId(id)})
    if recipe:
        recipe_collection.delete_one({"_id": ObjectId(id)})
        return True
Enter fullscreen mode Exit fullscreen mode

With the database CRUD functions in place, replace the content of app/api.py with:

from fastapi import FastAPI, Body
from fastapi.encoders import jsonable_encoder

from app.model import RecipeSchema, UpdateRecipeSchema
from app.database import save_recipe, get_all_recipes, get_single_recipe, update_recipe_data, remove_recipe

app = FastAPI()

@app.get("/", tags=["Root"])
def get_root() -> dict:
    return {
        "message": "Welcome to the okteto's app.",
    }

@app.get("/recipe", tags=["Recipe"])
def get_recipes() -> dict:
    recipes = get_all_recipes()
    return {
        "data": recipes
    }

@app.get("/recipe/{id}", tags=["Recipe"])
def get_recipe(id: str) -> dict:
    recipe = get_single_recipe(id)
    if recipe:
        return {
            "data": recipe
        }
    return {
        "error": "No such recipe with ID {} exist".format(id)
    }

@app.post("/recipe", tags=["Recipe"])
def add_recipe(recipe: RecipeSchema = Body(...)) -> dict:
    new_recipe = save_recipe(recipe.dict())
    return new_recipe

@app.put("/recipe", tags=["Recipe"])
def update_recipe(id: str, recipe_data: UpdateRecipeSchema)  -> dict:
    if not get_single_recipe(id):
        return {
            "error": "No such recipe exist"
        }

    update_recipe_data(id, recipe_data.dict())

    return {
        "message": "Recipe updated successfully."
    }

@app.delete("/recipe/{id}", tags=["Recipe"])
def delete_recipe(id: str) -> dict:
    if not get_single_recipe(id):
        return {
            "error": "Invalid ID passed"
        }


    remove_recipe(id)
    return {
        "message": "Recipe deleted successfully."
    }
Enter fullscreen mode Exit fullscreen mode

Update the api/model.py by removing the id field in the RecipeSchema model class:

class RecipeSchema(BaseModel):
    name: str = Field(...)
    ingredients: List[str] = Field(...)

    class Config:
        schema_extra = {
            "example": {
                "name": "Donuts",
                "ingredients": ["Flour", "Milk", "Sugar", "Vegetable Oil"]
            }
        }
Enter fullscreen mode Exit fullscreen mode

Testing The Database

With the database connection in place, start a mongod server to allow interactions with the database:

mongod --port 27017
Enter fullscreen mode Exit fullscreen mode

Next, test the POST route:

(venv)$ curl -X POST http://localhost:8080/recipe -d \
'{"name": "Donut", "ingredients": ["Flour", "Milk", "Butter"]}' \
-H 'Content-Type: application/json'
Enter fullscreen mode Exit fullscreen mode

Response:

{
  "id": "601fdcd82fbbf462d33a6e34"
}
Enter fullscreen mode Exit fullscreen mode

Test the GET routes:

  1. Return all recipes 
(venv)$  curl -X GET http://localhost:8080/recipe/2 -H 'Content-Type: application/json'
Enter fullscreen mode Exit fullscreen mode

Response:

{
  "data": [
    {
      "id": "601fdcd82fbbf462d33a6e34",
      "name": "Donut",
      "ingredients": [
        "Flour",
        "Milk",
        "Butter"
      ]
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode
  1. Return a single recipe 
(venv)$ curl -X GET http://localhost:8080/recipe/601fdcd82fbbf462d33a6e34 -H 'Content-Type: application/json'
Enter fullscreen mode Exit fullscreen mode

Response:

{
  "data": {
    "id": "601fdcd82fbbf462d33a6e34",
    "name": "Donut",
    "ingredients": [
      "Flour",
      "Milk",
      "Butter"
    ]
  }
}
Enter fullscreen mode Exit fullscreen mode

Test the UPDATE route:

(venv)$ curl -X PUT "http://0.0.0.0:8080/recipe?id=601fdcd82fbbf462d33a6e34" -H  "accept: application/json" -H  "Content-Type: application/json" -d "{\"name\":\"Buns\",\"ingredients\":[\"Flour\",\"Milk\",\"Sugar\",\"Vegetable Oil\"]}"
Enter fullscreen mode Exit fullscreen mode

Response:

{
  "message": "Recipe updated successfully."
}
Enter fullscreen mode Exit fullscreen mode

Lastly, test the DELETE route:

(venv)$ curl -X DELETE "http://0.0.0.0:8080/recipe/601fdcd82fbbf462d33a6e34" -H  "accept: application/json"
Enter fullscreen mode Exit fullscreen mode

Response:

{
  "message": "Recipe deleted successfully."
}
Enter fullscreen mode Exit fullscreen mode

Redeploying to Okteto

Okteto eases the stress of deployment and subsequent redeployment by allowing us to update and upgrade existing applications from the stack file. In the previous post, we created an okteto stack manifest to deploy our fastAPI service. We are now going to update it also include a mongodb instance as part of the deployment:

name: fastapi-crud
services:
  fastapi:
    public: true
    image: okteto.dev/fastapi-crud:latest
    build: .
    replicas: 1
    ports:
      - 8080
    resources:
      cpu: 100m
      memory: 128Mi
  mongodb:
    image: bitnami/mongodb:latest
    ports:
      - 27017
    resources:
      cpu: 100m
      memory: 128Mi
    volumes:
      - /bitnami/mongodb
Enter fullscreen mode Exit fullscreen mode

In the code above, you added another service, mongodb, to house a MongoDB container from bitnami. It's configured to expose MongoDB's default port 27017 is exposed. This container will only be accessible from your namespace, and it's configured with a persistent volume /bitnami/mongodb to ensure that data can be retrieved when the application restarts.

Under the fastapi service, add an environment heading containing the DB_HOST the database file reads using the decouple library:

environment:
  - DB_HOST=mongodb://mongodb:27017
  - secret=dev
Enter fullscreen mode Exit fullscreen mode

With the stack file updated, deploy it using the command:

(venv)$ okteto stack deploy --build
Enter fullscreen mode Exit fullscreen mode

Log on to your Okteto Dashboard. Notice that your application now includes an instance of MongoDB alongside your application:

Dashboard

Test the recipe route by replacing deployedapp from previous requests with the live application URL. From your terminal, run the command:

(venv)$ curl -X POST "https://fastapi-youngestdev.cloud.okteto.net/recipe" -H  "accept: application/json" -H  "Content-Type: application/json" -d "{\"name\":\"Donuts\",\"ingredients\":[\"Flour\",\"Milk\",\"Sugar\",\"Vegetable Oil\"]}"
Enter fullscreen mode Exit fullscreen mode

The response sent out is:

{
  "id": "601fe6deaa1a27fbcb9a60fb"
}
Enter fullscreen mode Exit fullscreen mode

Committing Changes to Git

With the changes confirmed and tested, commit all the changes in the application to the branch:

git add .
git commit -m "Added MongoDB to the recipe application"
Enter fullscreen mode Exit fullscreen mode

Push the commited changes:

git push -u origin mongo-crud
Enter fullscreen mode Exit fullscreen mode

Conclusion

In this article, you modified your fastAPI service to use a real database. You then added the database service to your okteto stack manifest and deployed the changes with one command. Finally, you tested the changes end to end, ensuring that they work as expected. The final version of the code is available on our Github repository.

Create your free Okteto account today and begin developing your new application with one click.

💖 💪 🙅 🚩
kvng_zeez
Abdulazeez Abdulazeez

Posted on April 20, 2021

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related

Adding A Database to Your Application Using Okteto Stacks
python Adding A Database to Your Application Using Okteto Stacks

April 20, 2021