One challenge with live coding on Twitch is maintaining continuity for viewers. Longer projects are can be especially difficult for viewers to follow along. To help with this I'm putting together these quick cliff notes about what was covered on previous streams along with resources I found helpful that viewers can quickly reference without having to rewatch an entire stream. I also count myself as a viewer and doing this exercise helps me stay organized by providing a quick way to recap what was covered previously. So here we go!

Project Overview

Before getting into the stream summary here is a little background on the project I'm working on. As my wife and I are planning for the birth of our first child I wanted to create a unique gift for her to remind her of all the love and support she has around her. In my previous streaming project I built a conversational voice search into the Peloton API, so I wanted to incorporate a voice component here. My first thought was to setup a Twilio voicemail box for people to leave greetings and words of encouragement for her.

After some more consideration I wanted to expand on that idea to include something tangible representing those voicemail messages. Years ago I was intrigued by fractals and remember a woman who was creating t-shirt designs based on fractals. Change your inputs a little and you get a new distinctive output. I began thinking could I use word vectors as input and generate a unique piece of algorithmic art representing all the messages? Over the past month I've been tinkering with generative art using word vectors as input so the art encodes real meaning in an abstract way.

To date I've been using transcripts of the My Favorite Murder podcast for testing. An example image is below where the black line represents Georgia and the red line is Karen. The lines at the bottom depict the podcast name and their tagline "Stay Sexy and Don't Get Murdered". Not knowing anything about generative art (or much about art in general) I'm pretty happy with the result so far. More tinkering is needed, but I like the general style of embedding meaning into the image without that meaning being obvious. There's still bunches to learn so if this project sounds interesting feel free to follow along on Twitch!

Stream Notes

In previous streams I started learning the pycairo library to draw lines of custom script using word vectors from podcast transcripts as inputs into the algorithm (like the one above). The colors in these pictures, however, were hardcoded. My goal in this stream was to use color to make the pictures more interesting. Specifically, I wanted to show a transition from one color to another. Each line of the picture would show a step in the color transition.

After some Googling I was a little surprised there isn't an existing Python library to perform the color transition from one color to the next. Maybe the problem isn't that novel to warrant its own library or I'm just bad at Google. Regardless, I decided to write my own function for the transition.

I made a couple assumptions. First, I'm assuming that the transition from color-to-color is linear. In reality this is a non-linear process, but this assumption vastly simplifies the problem and results in decent enough results to get started. Second, the pycairo function set_source_rgba function accepts floating point numbers for the RGB values so the examples here will using floating point values as well.

Given those assumptions the approach I took was to create a function color_diff to calculate how much each RGB value needs to change given a starting color, an ending color, and the number of steps the transition should occur within. More steps results in a smoother transition where fewer will be a more abrupt change. The resulting function looked like this:

def color_diff(color1, color2, n_steps):
    """Calculate the RGB differences between colors based on the number of steps
    the transition will occur within."""
    diff = [0, 0, 0]

    for i in range(len(color1)):
        diff[i] = (color2[i] - color1[i]) / n_steps

    return diff

For each RGB value the value of color1 is subtracted from color2 then divded by the total number of steps to be made. The result is a list with the values of how much the color needs to change at each step to transition to the desired destination color.

Now the function needs to be applied. The example below uses the color_diff function to adjust the brush color with every line drawn. The brush_color is initialized to be equal to the starting color. After the first line is drawn brush_color is incremented by color_step to slightly change the color.

The code is below. For this example straight lines are drawn evenly spaced along the y-axis. Try changing the value of n to see how the color gradient changes when you increase or decrease the number of lines you draw. You can change the values of colors as well to see how the gradients change.

import cairo
from transition import color_diff


# Setup the pycairo Surface.
WIDTH, HEIGHT = 600, 400
surface = cairo.ImageSurface(cairo.FORMAT_ARGB32, WIDTH, HEIGHT)
ctx = cairo.Context(surface)
ctx.scale(WIDTH, HEIGHT)  # Normalizing the canvas
ctx.set_source_rgba(0.95, 0.95, 0.95, 1)
ctx.paint()

start_color = [1, 0.38, 0.38]
end_color = [0, 0, 0]

# Number of lines in the image.
n = 5

# Calculate offset values to space lines on the Surface.
border_padding = 0.05
y_offset = ((1 - (border_padding * 2)) / n)

# Calculate color transition value for each line.
color_step = color_diff(start_color, end_color, n-1)

# Set the initial brush color as the start color.
brush_color = start_color

# Iteratively draw the lines.
x_start = 0.10
x_end = 0.90
for z in range(n):
    # Change brush color based on the color_step.
    if z > 0:
        brush_color[0] += color_step[0]
        brush_color[1] += color_step[1]
        brush_color[2] += color_step[2]

    # Define the line to draw.
    y = (y_offset * z) + border_padding
    ctx.move_to(x_start, y)
    ctx.line_to(x_end, y)

    # Style the brush.
    ctx.set_source_rgba(*brush_color, 1)
    ctx.set_line_width(0.002)

    # Draw the line.
    ctx.stroke()

# Save image.
surface.write_to_png("example_lines_transition.png")

Here is the expected output from the code:

Coming Up Next

The example here is a simple transition from one color to the next. One area to expand this is to allow user's to provide an entire color palette and transition through colors included in the palette.

On the next stream we're going to setup a Twilio voice mailbox for friends and family to call. We're going to setup a phone number for the voice mailbox and then look at the code to extract transcripts from messages left in the mailbox that can be used as input into an image. Come follow along and let's build something together!

Here is where you can find me on Twitch: https://www.twitch.tv/mattyvcodes

Stream Resources

On the stream I mentioned HuggingFace model to draw images from text you provided. I couldn't find the source during the stream, but with some digging I dredged it up! Here is DALL-E! Really neat application to play around with.

For this problem I found this SO answer helpful to think through potential approaches to the problem of color transitions. As I mentioned the solution I walk through in this post is a good starting point, but a more robust solution would likely consider that the color transitions are not going to be linear as I assume here.

Unrelated to this specific stream I am extremely grateful for all the folks who have shared their own repos of generative art for inspiration. A few I have found extra helpful are below:

• https://github.com/erdavids/Generative-Art
• https://github.com/anaulin/generative-art
• https://github.com/JakobGlock/Generative-Art