In this post, I'm gonna show you how to manage python dependencies when using the Serverless Framework. We'll use Pipenv but you can also apply a similar workflow if using other tools like poetry.

I'm assuming you have already installed serverless framework and have set up your credentials.

Setup

Let's start by setting up our environment.

If you are on a mac(you should):

brew install pipenv

if you are using Windows(why?) or another OS you can check here

For the purpose of demonstration lets create a new project:

serverless create --template aws-python3 --path manage-dep-workflow

This generates a new python v3 project inside a folder named manage-dep-workflow

Development workflow

The first time you navigate to your project's folder, in this case:

cd manage-dep-workflow

you need to activate the pipenv shell:

pipenv shell

Now you can start adding dependencies to your project with:

pipenv install <package>

so let's add pytest as a dependency

pipenv install pytest

Because we didn't specify a version for the dependency the latest is installed. If we wanted to do that we could do:

pipenv install pytest~=6.0.0

After doing this, pipenv will create two files: one named Pipfile and another Pipfile.lock. If you are familiar with NPM these are similar to package.json and package-lock.json

Let's take a look at Pipfile:

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
pytest = "~=6.0.0"

[requires]
python_version = "3.8"

we can see the pytest library installed under the [packages] section. There is another section [dev-packages] intended for development only libraries like pytest for example, so if we want to move it we can do it manually or use the command:

pipenv install pytest~=6.0.0 --dev

Integrate Pipenv and Serverless Framework

Now that we know how to install libraries into our project the final step is to integrate Pipenv with the Serverless Framework so that when we deploy our application our dependencies are bundled into the lambda packages.

For this, we are going to use the plugin serverless-python-requirements

This plugin will automatically detect Pipfile and pipenv installed and generate a requirements.txt file.

Let's go ahead and install it:

sls plugin install -n serverless-python-requirements

This will add a new entry in your serverless.yml plugin section like this:

plugins:
  - serverless-python-requirements

To verify everything is working together we can run:

sls deploy

or

sls package

In your terminal you will get an output like this one:

Serverless: Generating requirements.txt from Pipfile...
Serverless: Parsed requirements.txt from Pipfile in /Users/melianr/dev/python/manage-dep-workflow/.serverless/requirements.txt...
Serverless: Installing requirements from /Users/melianr/Library/Caches/serverless-python-requirements/24891d0bfa66a0144c82d36d710b97b82ea9498d08e4b45957ab78e085a1fdf7_slspyc/requirements.txt ...

As you can see the Serverless Framework is effectively creating a requirements.txt file on the fly and making it available for packaging.

Conclusion

To get more information about Pipenv commands you can read the official doc and to see more configuration options for the Serverless Python Requirements you can read more here

About me

Raisel Melian’s Twitter
Raisel Melian’s Linkedin
Raisel Melian’s YouTube channel