This post was originally published on my blog

Introduction

After developing an interest in lo-fi hip-hop music videos on YouTube, I wanted to learn how I could create my own. You know the kind, infinitely-looping GIFs from iconic anime films accompanied by lo-fi beats to relax/study to. I had some audio files and a collection of GIFs, but I didn’t know how to combine them. Then, I found FFmpeg!

What's FFmpeg?

According to the FFmpeg website, FFmpeg is "a complete, cross-platform solution to record, convert and stream audio and video". Today, we'll be using it to create a music video from a GIF and an MP3!

Getting Started

Before we get started, I'd like to point out that I'm using a MacBook Pro running OSX Mojave 10.14.5. If you're using a different OS, you can install FFmpeg by visiting their Download page.

Today, we'll be using Homebrew, the popular package manager for OSX, to install FFmpeg.

Open your Terminal to get started!

If you don't have Homebrew, you can get it by running

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

After you've installed Homebrew, you can get FFmpeg by running

brew install ffmpeg

If the install fails with the strange error, "Xcode alone is not sufficient on Mojave", run the suggested command

xcode-select --install

and install the Command Line Developer Tools when prompted. Then, run

open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

and follow the installation guide. Finally, re-run the FFmpeg install command.

To verify that FFmpeg has been installed correctly, run

which ffmpeg

You should see something like

/usr/local/bin/ffmpeg

Creating the Video

Now that we have FFmpeg, we can get started!

Make sure you have a GIF and MP3 file saved in the same folder on your computer. I put my files in a Documents/music folder, but you can save them wherever you'd like.

Navigate into the folder where your files are using

cd path/to/your/files

In my case, this would look like

cd Documents/music

Finally, we can use the following command to create our music video!

ffmpeg \
-i song.mp3 \
-stream_loop -1 \
-i cool-gif.gif \
-codec:v libx264 \
-preset slow \
-tune animation \
-codec:a aac \
-pix_fmt yuv420p \
-shortest \
mv.mp4

Replace song.mp3 and cool-gif.gif with the names of your files!

I named my output file mv.mp4, but you can name yours whatever you'd like. After the video is created, it will be automatically saved to the current folder.

Okay, now we have our awesome music video, but what is FFmpeg actually doing?...

Command Breakdown

ffmpeg: Invokes the FFmpeg program

-i [file] : Indicates that the specified file will be used as an input

-stream_loop -1: Sets the number of times an input should be looped. In our case, we want our GIF to loop forever, so we set the loop count to -1

-codec:v libx264: Specifies that we want to use the libx264 library to compress our GIF with the H.264 codec. H.264 is a fancy video compression tool, used to shrink the file size of a video while retaining as much quality as possible. H.264 uses cool compression techniques like chroma subsampling and motion compensation, which we won't get into today, but if you're interested in learning more, I highly recommend this blog post by Sid Bala

-preset slow: This setting changes the video encoding speed, which changes the compression of our video. Video encoding is a CPU-intensive task, so a slower encoding speed takes longer, but will give us a higher quality video. Feel free to explore other preset options!

-tune animation: This option changes the video encoding settings based on the type of input we're using. In my case, I used a GIF from an anime, so animation seemed like the best choice

For more information about the H.264 codec, or to explore other preset and tuning options, see the H.264 Encoding Guide in the Resouces section below

-codec:a aac: Specifies that we'd like to use AAC for audio encoding. AAC is a very popular audio encoder, and generally achieves better sound quality compared to MP3 files

pix_fmt yuv420p: Specifies that the YUV pixel format should be used for our output video. YUV is a special color space, similar to RGB, and allows our video to be played with QuickTime and other video players. If you'd like to learn more about YUV, the Wikipedia Page is a great place to start

-shortest: Tells FFmpeg to finish encoding as soon as the shortest input encoding is finished. This ensures that the duration of the music video matches the duration of the input audio

mv.mp4: The name of the output file

Resources

FFmpeg Documentation

H.264 Encoding Guide

Conclusion

After working with FFmpeg, I learned more about audio and video encoding, pixel formats, and more! I hope you did too! It was really cool to explore audio and video encoding while making something fun. Thanks for reading! If you enjoyed this post, let me know on Twitter :~)

Thanks to Greg Harris for reading a draft of this post