Managing external libraries in AWS lambda functions
Mohammad Abdul Alim
Posted on July 3, 2021
The popularity of microservices is increasing everyday. The main reason behind this is that we do not need to take care of the server related configurations. Just write down your code and see the magic. Definitely aws lambda is one of the best microservices to perform varieties of tasks.
When we write a lambda function, it is quite obvious that we may need third party modules and those will not be present in that environment. So we need to add them to lambda. Now the question comes how we can manage them properly? The best answer would be using lambda layers. According to aws document:
A Lambda layer is a .zip file archive that can contain additional code or data. A layer can contain libraries, a custom runtime, data, or configuration files.
So the idea is that we can create a lambda layer and keep other third party libraries in a zipped file. We can create multiple layers as per our requirements and use the same layer to multiple lambda functions. Let us create lambda layers in both node & python runtime & use it in respective lambda functions.
Lambda layer using node
In order to add a node lambda layer we will have to keep the installed dependencies and files in a folder named nodejs
. Here the folder name has to be nodejs
so that aws lambda can identify the dependencies. Let us now build a layer which contains the node-fetch
module that creates http requests with node js.
- Create a folder named
nodejs
. - Go inside that folder.
- Open the path in terminal.
- Create a
package.json
file usingnpm init -y
. - Then install required dependencies. Here only
node-fetch
will be installed usingnpm i node-fetch
. - Inside the
nodejs
folder there will be one folder namednode_modules
and two files namedpackage.json
&package-lock.json
. Compress thenodejs
folder to a zipped file.
Now we are ready to create our lambda layer. Just go to the aws console Create layer section like the following screenshot. Name your layer, choose a runtime like Node.js 14.x
& upload the zipped file created. Thus our lambda layer will be created.
Now we will use this layer in our lambda function. Assuming that you know how to create a lambda function ( if you don't, no worries coz its quite simple), just add the layer you created to your lambda function. In our lambda function we will make a GET
request to an api & show the response in a json
format. Our lambda function code is:
const fetch = require("node-fetch");
exports.handler = async (event) => {
try {
const response = await fetch(event.url);
const json = response.json();
return json;
} catch(e) {
return e;
}
};
We will create a test event containing:
{
"url": "https://reqres.in/api/users"
}
Now if we hit the Test
button we should see a response like following:
{
"page": 1,
"per_page": 6,
"total": 12,
"total_pages": 2,
"data": [
{
"id": 1,
"email": "george.bluth@reqres.in",
"first_name": "George",
"last_name": "Bluth",
"avatar": "https://reqres.in/img/faces/1-image.jpg"
},
{
"id": 2,
"email": "janet.weaver@reqres.in",
"first_name": "Janet",
"last_name": "Weaver",
"avatar": "https://reqres.in/img/faces/2-image.jpg"
},
{
"id": 3,
"email": "emma.wong@reqres.in",
"first_name": "Emma",
"last_name": "Wong",
"avatar": "https://reqres.in/img/faces/3-image.jpg"
},
{
"id": 4,
"email": "eve.holt@reqres.in",
"first_name": "Eve",
"last_name": "Holt",
"avatar": "https://reqres.in/img/faces/4-image.jpg"
},
{
"id": 5,
"email": "charles.morris@reqres.in",
"first_name": "Charles",
"last_name": "Morris",
"avatar": "https://reqres.in/img/faces/5-image.jpg"
},
{
"id": 6,
"email": "tracey.ramos@reqres.in",
"first_name": "Tracey",
"last_name": "Ramos",
"avatar": "https://reqres.in/img/faces/6-image.jpg"
}
],
"support": {
"url": "https://reqres.in/#support-heading",
"text": "To keep ReqRes free, contributions towards server costs are appreciated!"
}
}
Lambda layer using python
Now we will create a lambda layer to perform the similar functionality in python. We will be installing necessary modules for requests
to make http request and fetch data from remote api. Now we have to do as following:
- Create directories like this
build/python/lib/python3.8/site-packages
. We must not make any spelling mistake otherwise we will encounter error while running the lambda. Another important thing is the version of python. You must use the same version of python everywhere. Here I am using 3.8 so while selecting runtime I must select Python 3.8 to avoid errors. - Now go inside the
site-packages
folder and installrequests
using pip like this where-t .
means install everything inside this folder. - Now come back to the
build
folder & compress thepython
folder to a zipped file. - Now create a lambda layer like previous way & upload the zipped file. Select
Python 3.8
as runtime. ```
mkdir build/python/lib/python3.8/site-packages
cd build/python/lib/python3.8/site-packages
pip install requests -t .
Now its time to write down the lambda function. After creating the lambda, code will be as following:
import json
import requests
def lambda_handler(event, context):
response = requests.get(event['url'])
data = response.json()
return data
Our test event will be same as before & after running the lambda we will get similar json response as output.
Posted on July 3, 2021
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.