Create AI-generated art via SMS with Replicate in Python

This blog post was written for Twilio and originally published on the Twilio blog.

It's fun to get creative! Read on to learn how to create AI-generated images via SMS using Replicate and Twilio Programmable Messaging.

Do you prefer learning via video more? Check out this TikTok summarizing this tutorial.

Prerequisites

A Twilio account - sign up for a free Twilio account here
A Twilio phone number with SMS capabilities - learn how to buy a Twilio Phone Number here
Replicate account to use the Stable Diffusion text-to-image model – make a Replicate account here
Python installed - download Python here
ngrok, a handy utility to connect the development version of our Python application running on your machine to a public URL that Twilio can access.

⚠️ ngrok is needed for the development version of the application because your computer is likely behind a router or firewall, so it isn’t directly reachable on the Internet. You can also choose to automate ngrok as shown in this article.

Replicate

Replicate offers a cloud API and tooling so you can more easily run machine learning models, abstracting away some lower-level AI concepts and handling infrastructure so you can focus more on your own applications. You can run open-source models that others have published, or package and publish your own, either publicly or privately.

Configuration

Since you will be installing some Python packages for this project, you need to make a new project directory and a virtual environment.

If you're using a Unix or macOS system, open a terminal and enter the following commands:

mkdir replicate-prompt-to-image-sms 
cd replicate-prompt-to-image-sms 
python3 -m venv venv 
source venv/bin/activate 
pip install replicate flask twilio

If you're following this tutorial on Windows, enter the following commands in a command prompt window:

mkdir replicate-prompt-to-image-sms  
cd replicate-prompt-to-image-sms  
python -m venv venv 
venv\Scripts\activate 
pip install replicate flask twilio

Grab your default Replicate API Token or create a new one here. Run this command in the terminal in your current folder:

export REPLICATE_API_TOKEN={replace with your api token}

Now it's time to write some code to receive a prompt via SMS and return an AI-generated image!

Turn a Prompt into an AI-generated image via SMS with Replicate

Make a file called app.py and place the following import statements at the top.

import replicate
from flask import Flask, request
from twilio.twiml.messaging_response import MessagingResponse

Next, make a Flask app so your app can receive the inbound text message to your Twilio phone number. That string is then passed to Replicate 's stable diffusion model to generate photo-realistic images given that prompt.

app = Flask(__name__)
@app.route('/sms', methods=['GET', 'POST'])
def sms():
    resp = MessagingResponse()
    inb_msg = request.form['Body'].lower().strip()
    output = replicate.run(
        "stability-ai/stable-diffusion:27b93a2413e7f36cd83da926f3656280b2931564ff050bf9575f1fdf9bcd7478",
        input={"prompt": inb_msg}
        )
    res_prompt = "prompt was " + inb_msg
    msg = resp.message(res_prompt)
    img = output[0]
    msg.media(img)
    return str(resp)

if __name__ == "__main__":
    app.run(debug=True)

On the command line, run python app.py to start the Flask app. Now it's time to set up a Twilio phone number so you can text it a prompt and get an AI-generated image back!

Configure a Twilio Number for the SMS Chatbot

Now, your Flask app will need to be visible from the web so Twilio can send requests to it. ngrok lets you do this. With ngrok installed, run ngrok http 5000 in a new terminal tab in the directory your code is in.

You should see the screen above. Grab that ngrok Forwarding URL to configure your Twilio number: select your Twilio number under Active Numbers in your Twilio console, scroll to the Messaging section, and then modify the phone number’s routing by pasting the ngrok URL with the /sms path in the textbox corresponding to when A Message Comes In as shown below:

Click Save and now your Twilio phone number is configured so that it maps to your web application server running locally on your machine and your application can run. Text a prompt of a picture you'd like to generate to your Twilio number and you should get an AI-generated image of whatever you sent over SMS!

Instead of ngrok, you could alternatively use another tunneling service like Tailscale–my teammate Rishab reviewed this post, and kept getting a 403 error with ngrok. The solution was to switch port numbers with ngrok–it seems that Tailscale has more privileges to the network settings compared to ngrok. You can view the complete code on GitHub here.

What's Next for Replicate and Twilio Programmable Messaging?

There is so much fun for developers to have around building with LLMs! Replicate offers so many models to play with like sdxl, a text-to-image generative AI model that creates beautiful 1024x1024 images, or you can fine-tune your own. I can't wait to see what you build with AI–let me know online what you're working on!

Blog