🔊 Adding Sound FX to a Ratatui Game

In this post, we look at Ratatui audio with Rodio. I have been building a text-based User Interface (TUI) game in Ratatui. Ratatui is Rust tooling for Terminal apps. This is the third post in the series. I wrote the initial post once I had put together a minimal viable product, and outlined some next steps in that post. In the last post, I added fireworks to the victory screen by painting in a Ratatui canvas widget.

As well as fireworks, another enhancement that I identified (in the first post) was sound effects. I have built Rust games with sound before, but relied on the game’s tooling to add manage loading the audio assets and playing them (Macroquad and Bevy, for example, make this seamless). So, the first step was going to be finding a crate to play MP3 audio. I discovered Rodio, which worked well, and was quick to get going with.

In the rest of this post, I talk about the Rodio integration, and some next steps for the game. There is a link to the latest project repo, with full code further down.

🧱 Ratatui Audio with Rodio: What I Built

I didn’t really want background music, just sound effects to play when the player starts the challenge, and then to provide audio feedback if their latest solution attempt was good or perfect. Finally, I wanted to play some audio when each firework on the victory screen ignited. I found some quite small wave files for each of these, and converted them to MP3s.

About Rodio

Rodio uses a number of lower level crates under the hood, simplifying adding audio via their single higher level API. These, lower-level, crates include:

• cpal for playback;
• symphonia for MP4 and AAC playback;
• hound for WAV playback;
• lewton for Vorbis playback; and
• claxon for FLAC playback.

I just needed MP3s, so disabled default features, and only added symphonia-mp3 back in the project Cargo.toml, to keep the binary size in check:

[package]
name = "countdown-numbers"
version = "0.1.0"
edition = "2021"
license = "BSD-3-Clause"
repository = "https://github.com/rodneylab/countdown-numbers"
# ratatui v0.26.3 requires 1.74.0 or newer
rust-version = "1.74"
description = "Trying Ratatui TUI 🧑🏽‍🍳 building a text-based UI number game in the Terminal 🖥️ in Rust with Ratatui immediate mode rendering."

[dependencies]
num_parser = "1.0.2"
rand = "0.8.5"
ratatui = "0.27.0"
rodio = { version = "0.18.1", default-features = false, features = ["symphonia-mp3"] }

🐎 Adding Rodio

The Rodio docs give a couple of examples for getting going. Those examples read and decode an ogg file from a local folder within the project. This makes uses of the Rodio decoder struct, either appending it to a Rodio sink, or playing decoder output directly on an output stream. Either way, the audio is decoded and played straight away.

That setup works well for longer files, played once. For the game, I have a few sound effects that might be played dozens of times. It seemed a little extravagant to read from a file and decode each time I needed to play the same sound effect. Luckily, I found a Stack Overflow post with an alternative approach, letting you buffer the decoder output. Since the audio files were no bigger that 10 KB, I was happy to buffer them as the app starts up and keep them in memory.

Rust Code

I created a SoundEffects struct for holding all the sound effects, as there are only five of them. For an app with more, I would attempt a cleaner solution, but this approach keeps things simple for what I have.

use std::{fs::File, path::Path};

use rodio::{
    source::{Buffered, Source},
    Decoder,
};

pub struct SoundEffects {
    pub start: Buffered<Decoder<File>>,
    pub end: Buffered<Decoder<File>>,
    pub perfect: Buffered<Decoder<File>>,
    pub valid: Buffered<Decoder<File>>,
    pub firework: Buffered<Decoder<File>>,
}

fn buffer_sound_effect<P: AsRef<Path>>(path: P) -> Buffered<Decoder<File>> {
    let sound_file = File::open(&path)
        .unwrap_or_else(|_| panic!("Should be able to load `{}`", path.as_ref().display()));
    let source = Decoder::new(sound_file).unwrap_or_else(|_| {
        panic!(
            "Should be able to decode audio file `{}`",
            path.as_ref().display()
        )
    });

    source.buffered()
}

impl Default for SoundEffects {
    fn default() -> Self {
        SoundEffects {
            start: buffer_sound_effect("./assets/start.mp3"),
            end: buffer_sound_effect("./assets/end.mp3"),
            perfect: buffer_sound_effect("./assets/perfect.mp3"),
            valid: buffer_sound_effect("./assets/valid.mp3"),
            firework: buffer_sound_effect("./assets/firework.mp3"),
        }
    }
}

The default initializer for the SoundEffects struct just creates a buffered decoding for each of the effects, which can be called later from the app code.

In app code, I can then clone one of these buffers and add it to a sink to play it. For example in the main game loop:

fn run_app<B: Backend>(terminal: &mut Terminal<B>, app: &mut App) -> io::Result<()> {
    // ...TRUNCATED   

    let (_stream, stream_handle) = OutputStream::try_default().unwrap();
    let sink = Sink::try_new(&stream_handle).unwrap();
    let sound_effects = SoundEffects::default();

    loop {
        if event::poll(timeout)? {
            if let Event::Key(key) = event::read()? {
              // ...TRUNCATED

                match app.current_screen {
                  // ...TRUNCATED
                    CurrentScreen::PickingNumbers => match key.code {
                        KeyCode::Enter => {
                            if app.is_number_selection_complete() {
                                app.current_screen = CurrentScreen::Playing;
                                sink.append(sound_effects.start.clone());
                            }
                        }
                    }
                }
            }
        }
        // TRUNCATED...
    }
}

We initialize the sink ahead of the main loop, then play the buffered sound from within it (line 19), by calling sink.append(). sink.append() pushes the buffer into a queue and plays it immediately (if nothing is already playing), or waits until the last sound has finished before starting. That setup works here, and if you need to play sounds simultaneously, you can create multiple sinks. This setup also avoids borrow checker issues with consuming the File struct, which might arise when decoding the File within the main loop.

🙌🏽 Ratatui Audio with Rodio: Wrapping Up

In this Ratatui audio with Rodio post, I briefly ran through how I added audio to the Ratatui Countdown game. In particular, I talked about:

• why I added Rodio;
• why you might buffer Rodio to avoid borrow checker issues in a game loop; and
• how I buffered MP3 sound FX with Rodio.

I hope you found this useful. As promised, you can get the full project code on the Rodney Lab GitHub repo. I would love to hear from you, if you are also new to Rust game development. Do you have alternative resources you found useful? How will you use this code in your own projects?

🙏🏽 Ratatui Audio with Rodio: Feedback

If you have found this post useful, see links below for further related content on this site. Let me know if there are any ways I can improve on it. I hope you will use the code or starter in your own projects. Be sure to share your work on X, giving me a mention, so I can see what you did. Finally, be sure to let me know ideas for other short videos you would like to see. Read on to find ways to get in touch, further below. If you have found this post useful, even though you can only afford even a tiny contribution, please consider supporting me through Buy me a Coffee.

Finally, feel free to share the post on your social media accounts for all your followers who will find it useful. As well as leaving a comment below, you can get in touch via @askRodney on X (previously Twitter) and also, join the #rodney Element Matrix room. Also, see further ways to get in touch with Rodney Lab. I post regularly on Game Dev as well as Rust and C++ (among other topics). Also, subscribe to the newsletter to keep up-to-date with our latest projects.