(This article is [cross-posted](https://rwinslow.com/posts/use-flatbuffers-in-rust/) from [my blog](https://rwinslow.com).) The [FlatBuffers project](https://github.com/google/flatbuffers) is an extremely efficient schema-versioned serialization library. In this tutorial, you'll learn how to use it in Rust. To learn more about why we need yet another way to encode data, go read my post [Why FlatBuffers](https://rwinslow.com/posts/why-flatbuffers/). FlatBuffers is a serialization format from Google. It's really fast at reading and writing your data: much quicker than JSON or XML, and often faster than Google's other format, Protocol Buffers. It's schema-versioned, which means your data has integrity (like in a relational database). FlatBuffers supports thirteen programming languages: C++, C#, C, Dart, Go, Java, JavaScript, Lobster, Lua, PHP, Python, Rust, and TypeScript. This post will show you how to set up FlatBuffers and then use it in a demo Rust program. (Full disclosure: I maintain the Golang, Python, and Rust FlatBuffers codebases.) This tutorial has seven short parts: 1. [Install the FlatBuffers compiler](#section-installation) 2. [Create a new Cargo project (if needed)](#section-cargo-setup) 3. [Write a FlatBuffers schema definition](#section-schema) 4. [Generate Rust accessor code from the schema](#section-generate) 5. [Install the FlatBuffers Rust runtime library](#section-install-runtime) 6. [Write a demo Rust program to encode and decode example data](#section-demo) 7. [Learn more and get involved](#section-involved) If you'd like to see all of the code in one place, I've put the project up at [a GitHub repository](https://github.com/rw/rust_flatbuffers_example). ## 1. Install the FlatBuffers compiler First things first: let's install the compiler. The compiler is used only in development. That means you have no new system dependencies to worry about in production environments! ### Installation with Homebrew on OSX On my OSX system, I use [Homebrew](https://github.com/Homebrew/homebrew) to manage packages. To update the Homebrew library and install FlatBuffers, run: ``` sh $ brew update $ brew install flatbuffers ``` (As is usual on my blog, I indicate CLI input with the prefix `$`.) Personally, I like to install the latest development version from the official Git repository: ``` sh $ brew update $ brew install flatbuffers --HEAD ``` If successful, you will have the `flatc` program accessible from your shell. To verify it's installed, execute `flatc`: ``` sh $ flatc flatc: missing input files ... ``` ### Other installation methods If you'd like to install from source, install a Windows executable, or build for Visual Studio, head over to my post [Installing FlatBuffers](https://rwinslow.com/posts/how-to-install-flatbuffers/) for more. ## 2. Create a new Cargo project (if needed) (If you're adding FlatBuffers to an existing Rust project, you can skip this step.) Create a basic Cargo configuration with the following command: ``` sh $ cargo new rust_flatbuffers_example Created binary (application) `rust_flatbuffers_example` package ``` There will now be a directory called `rust_flatbuffers_example`. Change the current working directory to that: ``` sh $ cd rust_flatbuffers_example ``` Now, note that the directory contains the following files: ``` sh $ tree . |-- Cargo.toml `-- src `-- main.rs ``` Finally, check that the Cargo package is properly configured. Do this by running the example program that Cargo automatically generated: ``` $ cargo run --quiet Hello, world! ``` If you do not see this output, then please have a look at the [official documentation on setting up Rust and Cargo](https://doc.rust-lang.org/cargo/) to troubleshoot your configuration. ## 3. Write a FlatBuffers schema definition All data in FlatBuffers are defined by schemas. Schemas in FlatBuffers are plain text files, and they are similar in purpose to schemas in databases like Postgres. We'll work with data that make up user details for a website. It's a trivial example, but good for an introduction. Here's the schema: ``` // myschema.fbs namespace users; table User { name:string; id:ulong; } root_type User; ``` Place the above code in a file called `myschema.fbs`, in the root of your Cargo project. This schema defines `User`, which holds one user's `name` and `id`. The namespace for these types is `users` (which will be the generated Rust package name). The topmost type in our object hierarchy is the root type `User`. Schemas are a core part of FlatBuffers, and we're barely scratching the surface with this one. It's possible to have default values, vectors, objects-within-objects, enums, and more. If you're curious, go read the [documentation on the schema format](https://google.github.io/flatbuffers/md__schemas.html). ## 4. Generate Rust accessor code from the schema The next step is to use the `flatc` compiler to generate Rust code for us. It takes a schema file as input, and outputs ready-to-use Rust code. In the directory with the `myschema.fbs` file, run the following command: ``` sh $ flatc --rust -o src myschema.fbs ``` This will generate Rust code in a new file called `myschema_generated.rs` in the pre-existing `src` directory. Here's what our project looks like afterwards: ``` sh $ tree . |-- Cargo.lock |-- Cargo.toml |-- myschema.fbs `-- src |-- main.rs `-- myschema_generated.rs 1 directory, 6 files ``` Note that one file is generated for each schema file. A quick browse of `src/myschema_generated.rs` shows that there are three sections to the generated file. Here's how to think about the different function groups: + Type definition and initializer for reading `User` data ``` rust pub struct User { ... } pub fn get_root_as_user(buf: &[u8]) -> User { ... } pub fn get_size_prefixed_root_as_user(buf: &[u8]) -> User { ... } pub fn init_from_table(table: flatbuffers::Table) -> Self { ... } ``` + Instance methods providing read access to `User` data ``` rust pub fn name(&self) -> Option<&'a str> { ... } pub fn id(&self) -> u64 { ... } ``` + Functions used to create new `User` objects ``` rust pub fn create(_fbb: &mut flatbuffers::FlatBufferBuilder, args: &UserArgs) -> flatbuffers::WIPOffset { ... } ``` (Note that I've elided some of the lifetime annotations in the above code.) We'll use these functions when we write the demo program. ## 5. Install the FlatBuffers Rust runtime library The official FlatBuffers Rust runtime package is hosted on crates.io: [Official FlatBuffers Runtime Rust Library](https://crates.io/crates/flatbuffers). To use this in your project, add `flatbuffers` to your dependencies manifest in `Cargo.toml`. The file should now look similar to this: ``` toml [package] name = "rust_flatbuffers_example" version = "0.1.0" authors = ["rw "] edition = "2018" [dependencies] flatbuffers = "*" ``` I use `*` to fetch the latest package version. In general, you'll want to pick a specific version. You can learn more about this in the documentation for the [Cargo dependencies format](https://doc.rust-lang.org/cargo/guide/dependencies.html). ## 6. Write a demo Rust program to encode and decode example data Now, we'll overwrite the default Cargo "Hello World" program with code to write and read our FlatBuffers data. (We do this for the sake of simplicity, so that I can avoid explaining the Cargo build system here. To learn more about Cargo projects, head over to the [official documentation on Cargo project file layouts](https://doc.rust-lang.org/cargo/guide/project-layout.html).) ### Imports To begin, we will import the generated module using the `mod` statement. Place the following code in `src/main.rs`: ``` rust // src/main.rs part 1 of 4: imports extern crate flatbuffers; mod myschema_generated; use flatbuffers::FlatBufferBuilder; use myschema_generated::users::{User, UserArgs, finish_user_buffer, get_root_as_user}; ``` This usage of the `mod` keyword instructs the Rust build system to make the items in the file called `myschema_generated.rs` accessible to our program. The `use` statement makes two generated types, `User` and `UserArgs`, accessible to our code with convenient names. ### Writing FlatBuffer objects are stored directly in byte slices. Each Flatbuffers object is constructed using the generated functions we made with the `flatc` compiler. Append the following snippet to your `src/main.rs`: ``` rust // src/main.rs part 2 of 4: make_user function pub fn make_user(bldr: &mut FlatBufferBuilder, dest: &mut Vec, name: &str, id: u64) { // Reset the `bytes` Vec to a clean state. dest.clear(); // Reset the `FlatBufferBuilder` to a clean state. bldr.reset(); // Create a temporary `UserArgs` object to build a `User` object. // (Note how we call `bldr.create_string` to create the UTF-8 string // ergonomically.) let args = UserArgs{ name: Some(bldr.create_string(name)), id: id, }; // Call the `User::create` function with the `FlatBufferBuilder` and our // UserArgs object, to serialize the data to the FlatBuffer. The returned // value is an offset used to track the location of this serializaed data. let user_offset = User::create(bldr, &args); // Finish the write operation by calling the generated function // `finish_user_buffer` with the `user_offset` created by `User::create`. finish_user_buffer(bldr, user_offset); // Copy the serialized FlatBuffers data to our own byte buffer. let finished_data = bldr.finished_data(); dest.extend_from_slice(finished_data); } ``` This function takes a FlatBuffers `Builder` object and uses generated methods to write the user's name and ID. Note that the `name` string is created with the `bldr.create_string` function. We do this because, in FlatBuffers, variable-length data like strings need to be created *outside* the object that references them. In the code above, this is still ergonomic because we can call `bldr.create_string` inline from the `UserArgs` object. ### Reading FlatBuffer objects are stored as byte slices, and we access the data inside using the generated functions (that the `flatc` compiler made for us in `myschema_generated.rs`). Append the following code to your `src/main.rs`: ``` rust // src/main.rs part 3 of 4: read_user function pub fn read_user(buf: &[u8]) -> (&str, u64) { let u = get_root_as_user(buf); let name = u.name().unwrap(); let id = u.id(); (name, id) } ``` This function takes a byte slice as input, and initializes a FlatBuffer reader for the `User` type. It then gives us access to the name and ID values in the byte slice. ### The main function Now we tie it all together. This is the `main` function: ``` rust // src/main.rs part 4 of 4: main function fn main() { let mut bldr = FlatBufferBuilder::new(); let mut bytes: Vec = Vec::new(); // Write the provided `name` and `id` into the `bytes` Vec using the // FlatBufferBuilder `bldr`: make_user(&mut bldr, &mut bytes, "Arthur Dent", 42); // Now, `bytes` contains the serialized representation of our User object. // To read the serialized data, call our `read_user` function to decode // the `user` and `id`: let (name, id) = read_user(&bytes[..]); // Show the decoded information: println!("{} has id {}. The encoded data is {} bytes long.", name, id, bytes.len()); } ``` This function writes, reads, then prints our data. Note that `bytes` is the byte vector with encoded data. This is the serialized data you could send over the network, or save to a file. ### Running it ``` sh $ cargo run Arthur Dent has id 42. The buffer is 48 bytes long. ``` To recap, what we've done here is write a short program that uses generated code to write, then read, a byte slice in which we encoded data for an example User. This User has name "Arthur Dent" and ID 42. ## 7. Learn more and get involved FlatBuffers is an active open-source project, with backing from Google. It's Apache-licensed, and available for C++, C#, C, Dart, Go, Java, JavaScript, Lobster, Lua, PHP, Python, Rust, and TypeScript (with more languages on the way!). Here are some resources to get you started: + [GitHub repository](https://github.com/google/flatbuffers) + [Issue tracker](https://github.com/google/flatbuffers/issues) + [Official documentation](https://google.github.io/flatbuffers/) And my [additional blog posts on FlatBuffers](https://rwinslow.com/tags/flatbuffers/).

robert_winslow

Tutorial: Use FlatBuffers in Rust

Join Our Newsletter. No Spam, Only the good stuff.