Build an Image Search App with Koyeb, AWS Rekognition, and Algolia
Edouard Bonlieu
Posted on November 3, 2020
Introduction
This guide demonstrates how to create an image search application using the Koyeb Serverless Platform, AWS Rekognition to detect labels, and Algolia to index and search images labels.
Once you have completed this tutorial, you have a Next.js app up & running allowing you to search images by labels and upload new images. Each time an image is uploaded, a Koyeb function is triggered to
detect image labels using AWS Rekognition and index the result in Algolia.
You can preview the app we will build here.
- Koyeb is a serverless data processing platform offering a simple way to deploy and trigger functions and docker containers based on events.
- AWS Rekognition is an image detection service provided by Amazon Web Services that employs deep learning technology to analyze image content.
- Algolia is a powerful hosted search API that provides the resources & tools to create a fast and relevant search.
- Next.js is a React Framework offering many features such as hybrid static & server rendering, smart bundling, route pre-fetching, and more.
Requirements
To successfully follow and implement this tutorial, you need:
Steps
To create the image search application, you need to follow these steps:
- Create a Koyeb Store to Upload and Retrieve Files
- Generate AWS Rekognition Credentials
- Configure Algolia
- Deploy the Processing Function
Create a Koyeb Store to Upload and Retrieve Files
The first step is to create a Koyeb Store to upload and serve images. Koyeb natively provides an S3-compatible API allowing you to manage your data programmatically using any S3-compatible SDKs and tools.
We will use the Koyeb S3 API endpoint to upload and serve images in our app.
In this guide, we use a Koyeb Managed Store for simplicity, but you can connect your own Cloud Storage Provider to use it directly. You can refer to this documentation to get started with this method.
To create a new Koyeb Managed Store, go to the Koyeb Control Panel.
- Click Stores located in the menu on the left-side of the Koyeb Control Panel.
- On the Stores page, click Create Store.
- On the Store creation page, select Koyeb as the cloud service provider.
- Name your Store. Either keep the automatically generated name or enter the name you want to use for this Store.
- Click Create Store
We have now created a Koyeb Managed Store and can go to the next step.
Generate AWS Rekognition Credentials
As our Koyeb function uses AWS Rekognition to detect labels in images, we need to generate AWS Credentials to access the service.
Below, we generate new credentials scoped to the AWS Rekognition API and create new Secrets to store these credentials in the Koyeb console so our function can access them at runtime.
From the AWS Console
- Go to Identity and Access Management (IAM) dashboard
- Under Access management, click Users
- Click Add user
- Enter the username you want to use
- Select Programmatic Access
- Click Next: Permissions
- Select Attach existing policies directly
- Enter
AmazonRekognitionReadOnlyAccess
into the search bar - Click the box for
AmazonRekognitionReadOnlyAccess
- Click Next: Tags
- Add tags to remember where these credentials are used (Optional)
- Click Next: Review
- Review the information you have entered
- Click Create user
You have now successfully configured AWS Rekognition credentials. Save them in a secure place, we will need them in the next step.
From the Koyeb Console
- Go to your Koyeb Console.
- Click Secrets in the left-side menu.
- Click New on the Secrets page to create the first Secret. A modal appears.
- Enter the name of this Secret, for example:
aws-reko-access-key
. - Enter the AWS access key ID that you just created as the value.
- Click Submit.
- Click Create to configure another Secret; this one is for your AWS secret access key value.
- Enter the name of this Secret, for example:
aws-reko-secret-key
. - Enter the AWS secret key that you just created as the value.
- Click Submit.
We now have generated AWS Rekognition credentials and created the Koyeb Secrets required by our function to perform the label detection each time a new image is uploaded.
Configure Algolia
The next step is to configure Algolia by creating a new index to store image labels detect by AWS Rekognition.
Create an Index
- Click Indices in the left-side menu.
- Click Create index.
- A model appears and asks for an index name, for this demo we name it:
image-search
. - Click Submit.
Configure Koyeb Secrets
Now our index is created, we need to create new Koyeb Secrets to allow our function to index detected labels into Algolia at runtime.
From the Algolia console
- Click API Keys in the left-side menu.
- Copy the Application ID, Search-Only API Key, and the Admin API KEY values.
Make sure to copy these values in a safe place, we will need them in the next steps.
From the Koyeb Console
- Go to your Koyeb Console.
- Click Secrets in the left-side menu.
- Click New on the Secrets page to create a Secret. A modal appears.
- Enter the name of this Secret, for example:
algolia-app-id
. - Enter the Algolia Application ID that you just located the value.
- Click Submit.
- Click Create to configure another Secret; this one is for your Algolia API ID key value.
- Enter the name of this Secret, for example:
algolia-api-key
. - Enter the Algolia API key that you just located as the value.
- Click Submit.
From there, all the required configuration is ready. We can now deploy our function and view what happens.
Deploy the Processing Function
The best part starts now! Fork the GitHub repository containing the Koyeb processing function and demo app here.
The GitHub repository contains:
- A
koyeb.yaml
file: used to configure and deploy the functionimage-search.js
on Koyeb. Thekoyeb.yaml
file contains environment variables required by the function to run properly, the Koyeb Store the function must access to retrieve and process the image, and the event triggering the function. - A
image-search.js
file: The function to detect image labels and index them on Algolia. This function is triggered each time a new image is uploaded to the Koyeb Store.
All other files present in the repository are used to run the Next.js app. The app lets you explore the Koyeb Store images and upload new images that will be automatically processed by the function running on Koyeb.
Below is the repository structure:
## Koyeb folders and files
koyeb.yaml
koyeb-functions
└── image-search.js
## Next.js app folders & files
components
├── instant-search.js
└── uploader.js
pages
├── _app.js
├── api
│ ├── images
│ │ ├── [key].js
│ │ └── index.js
│ └── upload.js
└── index.js
public
├── favicon.ico
└── vercel.svg
styles
└── tailwind.css
package.json
node_modules
- Edit the
koyeb.yaml
Open the koyeb.yaml
and replace the REPLACE_ME_WITH_YOUR_KOYEB_STORE_NAME
with your Koyeb Store name. If you named the secrets as mentioned above, there is nothing else to edit.
functions:
- name: image-search
runtime: nodejs14
handler: koyeb-functions/image-search.handler
env:
AWS_ACCESS_KEY:
value_from_secret: aws-reko-access-key
AWS_SECRET_KEY:
value_from_secret: aws-reko-secret-key
ALGOLIA_APP_ID:
value_from_secret: algolia-app-id
ALGOLIA_API_KEY:
value_from_secret: algolia-api-key
ALGOLIA_INDEX:
value_from_secret: algolia-index
volumes:
- store: REPLACE_ME_WITH_YOUR_KOYEB_STORE_NAME
events:
- cloudevent:
expression: |
event.source == "koyeb.com/gateway" &&
event.subject == "REPLACE_ME_WITH_YOUR_KOYEB_STORE_NAME" &&
event.type.matches("s3:ObjectCreated:.*") &&
data.object.key.matches("([^\\s]+(\\.(?i)(jpe?g|png))$)")
- Create and deploy a new Stack
On the Koyeb console, click the Create button on the top-right and select Stack. You land on the Stack creation page. Choose GitHub repository as the way to manage the Stack and select the repository you previously forked in the dropdown. Click Create Stack.
- Go to the Store you created and upload some images
Go to the Store you created and upload some images. If you followed all steps, your Stack function is executed for each image uploaded, detect labels, and index them into Algolia.
In the Algolia control panel, you should see the list of images with their detected labels indexed.
Run the Next.js app
The final step is to run the Next.js app to showcase the end-to-end use-case.
Generate Koyeb S3 credentials
- In the Koyeb Console, click API in the left-side menu.
- Click New in the S3 Credentials section.
- Give a name to the credentials to remember what are they used for
- Copy the
access_key
andsecret_key
Configure the Next.js app
In the repository you previously forked, copy the env.example
to .env.local
, and edit the variable values with your own:
NEXT_PUBLIC_ALGOLIA_APP_ID=REPLACE_ME
NEXT_PUBLIC_ALGOLIA_SEARCH_API_KEY=REPLACE_ME
NEXT_PUBLIC_ALGOLIA_INDEX=REPLACE_ME
KOYEB_S3_ACCESS_KEY=REPLACE_ME
KOYEB_S3_SECRET_KEY=REPLACE_ME
KOYEB_STORE=REPLACE_ME
Run the app
In the terminal, run the following command to start the app:
-
yarn install
to install the project dependencies -
yarn start
to start the application
In your browser, open http://localhost:3000
, you land on a page listing your Koyeb Store images. You can search and upload new images. Each time a new image is uploaded, the function will be executed to detect labels and index the result in Algolia.
Posted on November 3, 2020
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.
Related
November 29, 2024