Queueing Emails With Python, Redis Queue And Twilio SendGrid

If you’re sending email from within a web application, you’ll want to do so asynchronously so you don’t block requests from completing. Especially if it takes additional (potentially slow) requests to construct the email content. There are several Python libraries available to help with asynchronous tasks, such as celery. Alternately, you can use asyncio if you’re on Python 3.4 or above.

Redis Queue is the friendliest to get started with. Today I’ll show you how to queue emails to be sent asynchronously with Redis Queue and Twilio SendGrid.

To keep things spicy, let’s use the Taco Fancy API to email a randomly generated taco recipe.

Getting Started

To follow along, you’ll need:

Python 3.5 or above installed
A free SendGrid account - sign up here
An email address, to test things out and make sure they’re working

First, create a new Python project. In case you need it, here’s a guide on general Python environment setup.

Create a file, build_mail.py, and open it in your editor of choice.

Diving Into The Tacolicious API

If you go to the taco API URL in the browser, you get some output that starts like this:

{"mixin": {"recipe": "### Green Chile Cabbage Salad with Seared Corn\nThis isn't a tradition, or even particularly traditional -- except in my apartment in Oakland, where I make this for myself every time I make pork tacos.
...

I’ve truncated the example output here as it’s quite long, but you get the idea.

We’re going to make an HTTP request to that API from our Python code, and then generate an email based on the response. Run pip install requests to locally install the requests library, which we’ll be using to make the API call.

The API returns the recipe in markdown format. We’re going to use a library called markdown2 to convert the markdown to HTML so it’s formatted correctly in the email. Run pip install markdown2 on the command line to install. Define a function, get_html_content, in the build_email file:

import requests
from markdown2 import Markdown

def get_html_content():
    url = 'http://taco-randomizer.herokuapp.com/random/'
    response = requests.get(url).json()

    taco_components = ['base_layer', 'mixin',
                       'condiment', 'seasoning', 'shell']
    html_content = f"<strong> Here's a taco recipe! </strong>\n"
    markdowner = Markdown()

    for component in taco_components:
       html_component = markdowner.convert(response.get(component).get('recipe'))
       html_content += f"<p>{html_component}</p>\n"
    return html_content

print(get_html_content())

Run this code by running python build_email.py on the command line. Your output will look something like this:

<strong> Here's a taco recipe! </strong>
<p><h1>Crock Pot Pulled Pork</h1>

<p>This is the base of my very favorite tacos. Added bonus is by cooking these all day in a slow cooker, your house smells <em>amazing</em>.</p>
...

Is your stomach growling yet?

It would be nice if our recipe had a title, so it didn’t feel like a random collection of components. Let’s make another function to build the title. Copy this code into build_email.py.

    title = ''
    taco_components_to_names = {'base_layer': ' with ',
                                'mixin': ' garnished with ',
                                'condiment': ' topped off with ',
                                'seasoning': ' and wrapped in delicious ',
                                'shell': '!'}
    for key, value in taco_components_to_names.items():
        title += f"{response.get(key).get('name')}{value}"

    return title

Now we’ll refactor our get_html_content function to call build_title:

   url = 'http://taco-randomizer.herokuapp.com/random/'
    response = requests.get(url).json()

    taco_components = ['base_layer', 'mixin',
                       'condiment', 'seasoning', 'shell']
    html_content = f"<strong> Here's a taco recipe! {build_title(response)} </strong>\n"
    markdowner = Markdown()

    for component in taco_components:
       html_component = markdowner.convert(response.get(component).get('recipe'))
       html_content += f"<p>{html_component}</p>\n"
    return html_content

  print(get_html_content())

Run python build_email.py script again from the command line. Check our your awesome new title.

<strong> Here's a taco recipe!  Baja Beer Battered Fish with Shredded Brussels Sprouts garnished with Mango Avocado Pico topped off with Kathy's Taco and Fajita Seasoning and wrapped in delicious bad-ass tortillas! </strong>

Next, we’ll add in some code to construct and send the email. Since we’re done with the recipe fetching part feel free to remove the print statements.

Setting up SendGrid

Create your API key from the SendGrid dashboard. Let’s call it “bulk email.”

After clicking “Create & View”, you’ll see your key. Before you close this dialog box, save the key in an environment variable named SENDGRID_API_KEY since you won’t be able to get that same key from the SendGrid dashboard again for security reasons.

Install the Sendgrid Python helper library with pip install sendgrid.

Generating the email content

Now we’ll write a function to actually send the email, using the SendGrid API.

import os
from sendgrid import SendGridAPIClient
from sendgrid.helpers.mail import Mail

def send_message(sendgrid_client, email):
    pass
    message = Mail(
        from_email=('tacobot@delicious.ceo', 'Taco Bot'),
        to_emails=email,
        html_content=get_html_content(),
        subject='Delicious tacos 🌮',
    )

    try:
        response = sendgrid_client.send(message)
        print(response.status_code)
        print(response.body)
        print(response.headers)
    except Exception as e:
        print(e)


sendgrid_client = SendGridAPIClient(os.environ.get('SENDGRID_API_KEY'))
email = ['test@example.com'] # your email goes here
send_message(sendgrid_client, email)

We only need to instantiate the SendGrid client once. We’ll enqueue a separate job for each email we want to send mail to. Thus we pass in sendgrid_client and email as parameters to our send_message function.

Run the script again from the command line:

202
b''
Server: nginx
Date: Tue, 17 Dec 2019 23:37:54 GMT
Content-Length: 0
Connection: close
X-Message-Id: yeFWKbW1R9Kf_8BeXSvFeg
Access-Control-Allow-Origin: https://sendgrid.api-docs.io
Access-Control-Allow-Methods: POST
Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl
Access-Control-Max-Age: 600
X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html

In your inbox, you should receive a mouthwatering recipe.

Adding Redis Queue

Redis is “an open source (BSD licensed), in-memory data structure store.” Before you install Redis Queue, you must install redis on the machine where your code will be running. Download the latest stable version by following the instructions here. Then start the Redis server by running src/redis-server in the directory where you downloaded Redis. You won’t need to do anything else with the Redis server, it just needs to stay running in the background.

Install Redis Queue by running pip install rq from the terminal in your Python project directory. Create another file in the same folder where the build_email script lives. Call it enqueue.py. We need another file because otherwise Redis Queue will throw an error message. We’ll migrate some of our code over to the new file.

from redis import Redis
from rq import Queue
import os

from build_email import send_message
from sendgrid import SendGridAPIClient

sendgrid_client = SendGridAPIClient(os.environ.get('SENDGRID_API_KEY'))

q = Queue(connection=Redis())

emails = ['example1@mail.com', 'example2@mail.com', 'example3@mail.com'] # replace these with your email addresses

for email in emails:
    q.enqueue(send_message, sendgrid_client, email)

You don’t need to use three email addresses here - you can use as many or as few as you’d like.

Before you run this code, you need to run the rqworker command in another terminal tab in the same folder where your project code lives. Don’t forget to activate your Python virtualenv on the new tab first.

After you start rqworker, try running python enqueue.py from the command line. Look on the terminal tab where you started rqworker and you should see some output like so:

17:48:16 default: Job OK (9c9d8195-0297-4528-8c33-7ada97ecf114)
17:48:16 Result is kept for 500 seconds
17:48:16 default: send_mail.send_message(<sendgrid.sendgrid.SendGridAPIClient object at 0x1019e7748>, 'tilde@thuryism.net') (98f72e19-f75d-4664-b4f5-c6c04db3edb3)
202
b''
Server: nginx
Date: Tue, 17 Dec 2019 01:48:17 GMT
Content-Length: 0
Connection: close
X-Message-Id: ZAceAObvT96H2ZDW3aJImQ
Access-Control-Allow-Origin: https://sendgrid.api-docs.io
Access-Control-Allow-Methods: POST
Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl
Access-Control-Max-Age: 600
X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html

Every email address on the list should receive a unique and delicious recipe. That’s it! You can check out the example code for this whole project on GitHub.

Conclusion

Today we learned how to queue emails for asynchronous sending with Twilio SendGrid and Redis Queue.

What else can you do with Redis Queue?

You can receive SMS notifications when the International Space Station flies above you.
You can print photos from the Mars Rover.
You can even generate Nintendo music.

Hungry? Go forth and queue some delicious tacos into your face. 🌮

Blog