The Hard Things About Rust
Moe Katib
Posted on June 20, 2023
Rust is a systems programming language that runs blazingly fast, prevents segfaults, and guarantees thread safety. While these features make Rust a powerful tool for systems programming, they also introduce several new concepts that may be unfamiliar to those coming from other languages.
In this comprehensive guide, "The Hard Things About Rust", we aim to shed light on these challenging aspects of Rust and make them accessible to both new and experienced programmers. We'll unravel these complex concepts, illustrating each one with concrete examples and real-world scenarios for better understanding.
Here's a glimpse of what we'll be covering:
Ownership: We'll start with the fundamental concept of Ownership in Rust. We'll explore what it means for a value to have an owner, how ownership can be transferred, and how Rust's Ownership model aids in memory management.
Borrowing and Lifetimes: Building on Ownership, we'll then delve into Borrowing and Lifetimes, two interconnected concepts that allow you to safely reference data.
Slices: We'll demystify Slices, a view into a block of memory, which are used extensively in Rust for efficient access to data.
Error Handling: Rust's approach to handling errors is unique and robust. We'll cover the
ResultandOptiontypes, and how they are used for elegant error handling.Concurrency: We'll dig into Rust's powerful yet complex concurrency model. We'll talk about threads, message passing, and shared state concurrency, among other things.
Advanced Types and Traits: We'll explore some of Rust's advanced types, like
Box,Rc,Arc. We'll also cover Traits and Trait Objects.Async/Await and Futures: As we move towards the advanced concepts, we'll explain Rust's async/await syntax and the Futures model for handling asynchronous programming.
The goal of this guide is not just to provide an overview of these topics, but to help you understand the rationale behind these concepts, how they work under the hood, and how you can effectively use them in your Rust programs.
Whether you're a Rust beginner looking to understand the language deeply or an intermediate Rustacean aiming to solidify your understanding of these complex concepts, this guide is for you. Let's embark on this journey to conquer the hard things about Rust!
Ownership
Ownership is a foundational concept in Rust. It is part of Rust's approach to memory safety and makes Rust unique among programming languages. Understanding Ownership is crucial for writing Rust programs, as many other Rust concepts, like Borrowing and Lifetimes, are built upon it.
What is Ownership?
In Rust, every value has a variable that's called its owner. There can only be one owner at a time. When the owner goes out of scope, the value will be dropped, or cleaned up.
Let's consider a simple example:
{
let s = "hello world"; // s is the owner of the &str "hello world"
} // s goes out of scope here, and the string is dropped
In the code above, the variable s is the owner of the string "hello world". Once s goes out of scope at the end of the block, the string is dropped and its memory is freed.
Moving Ownership
In Rust, the assignment operator = moves ownership from one variable to another. This is different from other languages where = copies the value.
Consider this example:
let s1 = String::from("hello");
let s2 = s1;
In the code above, s1 initially owns the string "hello". However, the line let s2 = s1; moves the ownership from s1 to s2. Now, s2 is the owner of the string "hello", and s1 is no longer valid. If you try to use s1 after this, Rust will give you a compile-time error.
Copy Trait
Some types in Rust implement the Copy trait. When such types are assigned to another variable, instead of moving the ownership, a copy of the value is made. All the integer and floating point types, boolean type, character type, and tuples of types implementing Copy trait are Copy.
Here is an example:
let x = 5;
let y = x;
In the code above, x is an integer, which implements the Copy trait. So, when we write let y = x;, it doesn't move the ownership. Instead, it copies the value from x to y.
Why Ownership?
The concept of Ownership enables Rust to make memory safety guarantees without needing a garbage collector. By enforcing that there can only be one owner of a value and that the value is cleaned up when the owner goes out of scope, Rust can prevent common programming errors like null or dangling pointers, double free, and data races.
Borrowing and Lifetimes: Safely Referencing Data in Rust
Borrowing and Lifetimes are two of the most distinctive features of Rust. Together, they empower Rust to guarantee memory safety and thread safety without a garbage collector. Let's explore these concepts in detail.
Borrowing
In Rust, we often let other parts of our code access a value without taking ownership over it. This is done through a feature called 'borrowing'. There are two types of borrows: shared borrows and mutable borrows.
Shared Borrow
A shared borrow allows an item to have multiple references. This is accomplished by using the & symbol in Rust. Let's take a look at an example:
fn main() {
let s1 = String::from("hello");
let len = calculate_length(&s1);
println!("The length of '{}' is {}.", s1, len);
}
fn calculate_length(s: &String) -> usize {
s.len()
}
In this code, calculate_length is borrowing s1 for temporary use. s1 is still owned by the main function, so we can use s1 again after the calculate_length call.
Mutable Borrow
A mutable borrow is when you want to allow the borrowed value to be changed. This is done using &mut in front of the variable. For example:
fn main() {
let mut s1 = String::from("hello");
change(&mut s1);
}
fn change(s: &mut String) {
s.push_str(", world");
}
Here, the change function is borrowing s1 and altering it. This is possible because s1 was mutably borrowed.
However, Rust has a rule that you can have either one mutable reference or any number of immutable references, but not both. This rule guarantees data races will never occur.
Let's break down the concept with a code example.
fn main() {
let mut s = String::from("hello");
let r1 = &s; // no problem
let r2 = &s; // no problem
println!("{} and {}", r1, r2);
// r1 and r2 are no longer used after this point
let r3 = &mut s; // no problem
println!("{}", r3);
}
In this example, the code works because even though r1 and r2 are in scope when r3 is created, they are not used after r3 is created. Rust's rules state (as mentioned before) that you can have either one mutable reference or any number of immutable references, but not both. But this only applies when the references are used.
Now, let's see an example where Rust's borrowing rules are violated:
fn main() {
let mut s = String::from("hello");
let r1 = &s; // no problem
let r2 = &s; // no problem
let r3 = &mut s; // PROBLEM! // cannot borrow `s` as mutable because it is also borrowed as immutable
println!("{}, {}, and {}", r1, r2, r3);
}
In this case, we have r1 and r2 which are immutable references, and r3 which is a mutable reference. We're trying to use all of them at the same time, which violates Rust's borrowing rules, hence the compiler will throw an error.
This rule prevents data races at compile time.
Lifetimes
Lifetimes are Rust's way of ensuring that all borrows are valid. The main point of lifetimes is to prevent dangling references. A dangling reference occurs when we have a reference to some data, and that data gets dropped before the reference does.
In Rust, the compiler uses lifetimes to ensure that these kinds of errors cannot occur. Here's an example:
fn longest<'a>(x: &'a str, y: &'a str) -> &'a str {
if x.len() > y.len() {
x
} else {
y
}
}
In this function, 'a is a lifetime parameter that says: for some lifetime 'a, take two parameters, both are string slices that live at least as long as 'a, and return a string slice that also will last at least as long as 'a.
This is a bit abstract, so let's consider a specific example:
fn main() {
let string1 = String::from("long string is long");
{
let string2 = String::from("xyz");
let result = longest(string1.as_str(), string2.as_str());
println!("The longest string is {}", result);
}
}
Here, the lifetime of string1 is longer than that of string2, so when result is used in the println!, it won't be referring to string2, ensuring that we don't have a dangling reference.
In conclusion, borrowing and lifetimes are two sides of the same coin that make Rust safe and efficient. They allow Rust to ensure safety and concurrency at compile time. Understanding them is key to mastering Rust.
Slices: A View into Sequences in Rust
Rust provides a way to reference a contiguous sequence, or a part of a collection, rather than the whole collection itself. This is done through a feature called 'slices'.
Understanding Slices
A slice represents a reference to one or more contiguous elements in a collection rather than the whole collection. Here's an example of a slice:
fn main() {
let s = String::from("hello world");
let hello = &s[0..5];
let world = &s[6..11];
println!("{} {}", hello, world);
}
In this code, hello and world are slices of s. The numbers [0..5] and [6..11] are range indices that state "start at index 0 and continue up to, but not including, index 5" and "start at index 6 and continue up to, but not including, index 11", respectively. If we run this program, it will print hello world.
String Slices
A string slice is a reference to part of a String, and it looks like this:
let s = String::from("hello world");
let hello = &s[0..5];
let world = &s[6..11];
Here hello and world are slices of the string s. You can create a slice by using a range within brackets by specifying [starting_index..ending_index], where starting_index is the first position in the slice and ending_index is one more than the last position in the slice.
Array Slices
Just like strings, we can also slice arrays. Here's an example:
fn main() {
let a = [1, 2, 3, 4, 5];
let slice = &a[1..3];
println!("{:?}", slice);
}
Here, slice will be a slice that contains 2, 3, which are the second and third elements of the array a.
The Benefit of Slices
The power of slices is that they let you refer to a contiguous sequence without copying the sequence into a new collection. This is a more efficient way to let functions access a portion of a collection.
Error Handling in Rust
Error handling is a fundamental part of any programming language, and Rust is no exception. It recognizes the inevitability of errors in software and provides robust mechanisms to handle them effectively. The design of Rust's error handling mechanisms demands that developers acknowledge and handle errors explicitly, thereby making programs more robust and preventing many issues from reaching production environments.
Rust categorizes errors into two major types: recoverable and unrecoverable errors. Recoverable errors are often the result of operations that can fail under normal conditions, such as trying to open a file that does not exist. In such cases, we typically want to inform the user about the error and retry the operation or carry on with the program's execution in a different manner.
On the other hand, unrecoverable errors are usually indicative of bugs in your code, such as trying to access an array beyond its bounds. These kinds of errors are severe enough to warrant stopping the program immediately.
Interestingly, Rust does not use exceptions, a common error handling mechanism in many languages. Instead, it provides two constructs: Result<T, E> and the panic! macro, for handling recoverable and unrecoverable errors respectively.
panic! Macro
The panic! macro in Rust is used to halt the execution of the program immediately. It is typically used when the program encounters a situation that it doesn't know how to handle or when it has reached a state it should never have reached. These scenarios often represent bugs in the program. When panic! is called, an error message is printed to the standard error output, and the program is terminated.
You can call panic! with a simple string message, or use it with format strings, similar to println!. The message you pass to panic! becomes the panic payload and is returned as part of the error message when the program crashes. For example:
panic!();
panic!("this is a terrible mistake!");
panic!("this is a {} {message}", "fancy", message = "message");
std::panic::panic_any(4); // panic with the value of 4 to be collected elsewhere
If panic! is called in the main thread, it will terminate all other threads and end your program with exit code 101.
Result<T, E> Enum
Rust's approach to handling recoverable errors is encapsulated in the Result<T, E> enum. Result is a generic enum with two variants: Ok(T) representing a successful result, and Err(E) representing an error. The power of Result lies in its explicit nature; it forces the developer to handle both the success and failure cases, thereby avoiding many common error handling pitfalls.
Rust provides several methods for handling Result values, the most notable of which is the ? operator. The ? operator can be appended to the end of a function call that returns a Result. If the function was successful and returned Ok(T), the ? operator unwraps the value T and the program continues. If the function encountered an error and returned Err(E), the ? operator immediately returns from the current function and propagates the error up the call stack.
enum Result<T, E> {
Ok(T),
Err(E),
}
This definition signifies that a function that returns a Result can either be successful (Ok) and return a value of type T, or it can fail (Err) and return an error of type E.
Here is an example of a function that returns a Result:
use std::num::ParseIntError;
fn parse_number(s: &str) -> Result<i32, ParseIntError> {
match s.parse::<i32>() {
Ok(n) => Ok(n),
Err(e) => Err(e),
}
}
let n = parse_number("42");
match n {
Ok(n) => println!("The number is {}", n),
Err(e) => println!("Error: {}", e),
}
In this example, parse_number tries to parse a string into an integer. If successful, it returns the number inside Ok, otherwise it returns the error inside Err. The match statement is used to handle both possible outcomes of the Result.
Option
The Option enum is similar to Result, but it's used when a function could return a value or nothing at all, not an error. It's defined as:
enum Option<T> {
Some(T),
None,
}
Here is an example of a function that returns an Option:
fn find(array: &[i32], target: i32) -> Option<usize> {
for (index, &item) in array.iter().enumerate() {
if item == target {
return Some(index);
}
}
None
}
let array = [1, 2, 3, 4, 5];
match find(&array, 3) {
Some(index) => println!("Found at index {}", index),
None => println!("Not found"),
}
In this example, the find function tries to find a number in an array. If it's found, the function returns Some(index), where index is the position of the number in the array. If it's not found, the function returns None.
Both Result and Option provide various useful methods to work with these types. For instance, unwrap can be used to get the value inside Ok or Some, but it will panic if the Result is Err or the Option is None. As a safer alternative, unwrap_or and unwrap_or_else can be used to provide a default value or a fallback function respectively.
let x = Some(2);
assert_eq!(x.unwrap(), 2);
let x: Option<u32> = None;
assert_eq!(x.unwrap_or(42), 42);
let x: Result<u32, &str> = Err("emergency failure");
assert_eq!(x.unwrap_or_else(|_| 42), 42);
In general, Result and Option are powerful tools in Rust for error handling and for representing the absence of a value. They make your code more explicit about possible failure or null cases, helping to prevent many common programming errors.
A Look Into Concurrency
Concurrency in Rust is achieved through a number of mechanisms, including threads, message passing, and shared state. Let's explore each of these in turn.
1. Threads
Rust has a std::thread module that allows you to create new threads and work with them in a system-independent manner. Here is a simple example of creating a new thread:
use std::thread;
use std::time::Duration;
fn main() {
thread::spawn(|| {
for i in 1..10 {
println!("hi number {} from the spawned thread!", i);
thread::sleep(Duration::from_millis(1));
}
});
for i in 1..5 {
println!("hi number {} from the main thread!", i);
thread::sleep(Duration::from_millis(1));
}
}
In this example, we're creating a new thread with thread::spawn and passing it a closure that includes the instructions for the new thread. The main thread and the new thread print out their messages independently, sleeping for a millisecond between each message.
2. Message Passing
Rust provides a message-passing concurrency model inspired by the language Erlang. Message passing is a way of handling concurrency where threads or actors communicate by sending each other messages containing data.
In Rust, you can create a channel using the std::sync::mpsc module (mpsc stands for multiple producers, single consumer). Here's an example:
use std::sync::mpsc;
use std::thread;
fn main() {
let (tx, rx) = mpsc::channel();
thread::spawn(move || {
let val = String::from("hi");
tx.send(val).unwrap();
});
let received = rx.recv().unwrap();
println!("Got: {}", received);
}
In this example, we're creating a channel with mpsc::channel, then moving the transmission end (tx) into a new thread. That thread sends a message ("hi") into the channel, and then we wait to receive the message in the main thread and print it out.
3. Shared State
Rust also provides a way to share state between threads in a safe manner using Mutexes. A Mutex provides mutual exclusion, meaning that only one thread can access the data at any given time. To access the data, a thread must first signal that it wants access by asking the mutex to lock. Here's an example:
use std::sync::{Mutex, Arc};
use std::thread;
fn main() {
let counter = Arc::new(Mutex::new(0));
let mut handles = vec![];
for _ in 0..10 {
let counter = Arc::clone(&counter);
let handle = thread::spawn(move || {
let mut num = counter.lock().unwrap();
*num += 1;
});
handles.push(handle);
}
for handle in handles {
handle.join().unwrap();
}
println!("Result: {}", *counter.lock().unwrap());
}
In this example, we're creating a counter inside a Mutex, then sharing it between multiple threads using an atomic reference count (Arc). Each thread locks the mutex, increments the counter, then releases the lock.
This is a high-level overview. Rust's concurrency model is quite powerful and flexible, and it provides many features to ensure that concurrent code is safe from data races and other common concurrency problems.
Advanced Types and Traits
Box
Box is a smart pointer that points to data stored on the heap, rather than the stack. They're useful when you have a large amount of data to store or you want to ensure a specific variable isn't moved around in memory.
Box also has ownership. When a Box goes out of scope, the destructor is called and the heap memory is deallocated.
Here's a simple example:
let b = Box::new(5); // b is a pointer to a heap allocated integer
println!("b = {}", *b); // Output: b = 5
In this example, the variable b is a Box that owns an integer, 5, on the heap. The * operator is used to dereference the box, getting the value it points to.
Rc
Rc stands for Reference Counting. It's a smart pointer that allows for multiple owners by keeping track of the number of references to a value which determines when to clean up. Rc is used when we want to allocate some data on the heap for multiple parts of our program to read, and we can't determine at compile time which part will finish using the data last.
It's important to note that Rc is only for use in single-threaded scenarios. Here's a simple example:
use std::rc::Rc;
let original = Rc::new(5);
let a = Rc::clone(&original);
let b = Rc::clone(&original);
println!("original: {}, a: {}, b: {}", *original, *a, *b); // Output: original: 5, a: 5, b: 5
In this example, the variable original is an Rc that owns an integer, 5, on the heap. We can create multiple "clones" of this Rc (which are actually just new pointers to the same data, not full copies). When all of the Rcs go out of scope, the heap memory will be deallocated.
Arc
Arc is Atomic Reference Counting. It's the same as Rc, but safe to use in multithreaded contexts. It provides the same functionality as Rc, but uses atomic operations for its reference counting. This makes it safe to share between many threads, at the cost of a minor performance hit.
Here's an example:
use std::sync::Arc;
use std::thread;
let original = Arc::new(5);
for _ in 0..10 {
let original = Arc::clone(&original);
thread::spawn(move || {
println!("{}", *original);
});
}
In this example, we use an Arc to share a heap-allocated integer between multiple threads. Each thread gets a clone of the Arc (a new pointer to the data). When all Arcs go out of scope, the heap memory is deallocated.
These types offer more advanced ways to manage memory and data ownership in Rust, enabling more complex data structures and patterns. However, they also add complexity and can be harder to use correctly, so they should be used judiciously.
Traits
In Rust, a trait is a collection of methods defined for an unknown type: Self. They can access other methods declared in the same trait, and are a way to define shared, or common behavior. Think of traits as a way of defining interfaces that types can implement.
Consider this simple example:
trait Animal {
fn make_noise(&self) -> String;
}
struct Dog;
struct Cat;
impl Animal for Dog {
fn make_noise(&self) -> String {
String::from("Woof!")
}
}
impl Animal for Cat {
fn make_noise(&self) -> String {
String::from("Meow!")
}
}
In the above example, we've defined a trait Animal with a method make_noise. We then implement this trait for the Dog and Cat structs, providing their unique versions of the make_noise function. We can now call this function on any type that implements the Animal trait.
Clone and Copy Traits
Rust offers a number of predefined traits with specific behaviors. Two of these are the Clone and Copy traits.
The Clone trait allows for the explicit duplication of data. When you want to make a new copy of a type's data, you can call the clone method if the type implements the Clone trait.
#[derive(Clone)]
struct Point {
x: i32,
y: i32,
}
let p1 = Point { x: 1, y: 2 };
let p2 = p1.clone(); // p1 is cloned into p2
In this example, the Point struct implements the Clone trait, so we can create a duplicate of any Point instance using the clone method.
On the other hand, the Copy trait allows for implicit duplication of data. It is used when we want to be able to make shallow copies of values without worrying about ownership. If a type implements the Copy trait, an older variable is still usable after assignment.
#[derive(Copy, Clone)]
struct Simple {
a: i32,
}
let s1 = Simple { a: 10 };
let s2 = s1; // s1 is copied into s2
println!("s1: {}", s1.a); // s1 is still usable
In this example, Simple implements the Copy trait, allowing s1 to be copied into s2 and still remain usable afterwards.
However, a word of caution: not all types can be Copy. Types that manage a resource, like a String or a custom struct owning heap data, can't implement Copy trait. In general, if a type requires some special action when the value is dropped, it can't be Copy. This restriction protects against double-free errors, a common issue in languages with manual memory management.
Debug Trait
The Debug trait enables formatting of the struct data for output, typically used for debugging purposes. By default, Rust doesn't allow printing struct values. However, once the Debug trait is derived, you can use the println! macro with debug formatting ({:?}) to print out the struct values.
#[derive(Debug)]
struct Rectangle {
width: u32,
height: u32,
}
let rect = Rectangle { width: 30, height: 50 };
println!("rect is {:?}", rect);
In this example, Rectangle derives the Debug trait, allowing you to print out its value in the standard output.
PartialEq and Eq Traits
The PartialEq trait allows for comparison of type instances for equality and inequality. The Eq trait, which depends on PartialEq, signifies that all comparisons are reflexive, meaning if a == b and b == c, then a == c.
#[derive(PartialEq, Eq)]
struct Point {
x: i32,
y: i32,
}
let p1 = Point { x: 1, y: 2 };
let p2 = Point { x: 1, y: 2 };
println!("Are p1 and p2 equal? {}", p1 == p2);
In this example, Point derives the PartialEq and Eq traits, which allows for comparison of Point instances.
PartialOrd and Ord Traits
These traits enable comparison operations (<, >, <=, >=) on type instances. PartialOrd allows for partial ordering, where some values can be incomparable. On the other hand, Ord enables a full ordering between values.
#[derive(PartialOrd, Ord, PartialEq, Eq)]
struct Point {
x: i32,
}
let p1 = Point { x: 1 };
let p2 = Point { x: 2 };
println!("Is p1 less than p2? {}", p1 < p2);
In this example, Point derives the PartialOrd, Ord, PartialEq, and Eq traits. This allows for comparison of Point instances.
Default Trait
The Default trait allows for creation of default values of a type. It provides a function default which returns the default value of a type.
#[derive(Default)]
struct Point {
x: i32,
y: i32,
}
let p1 = Point::default(); // Creates a Point with x and y set to 0
In this example, Point derives the Default trait. This allows for creation of a Point instance with default values (0 in this case).
Async/Await and Futures
Futures
A Future in Rust represents a value that may not have been computed yet. They are a concept in concurrent programming that enables non-blocking computation: rather than waiting for a slow computation to finish, the program can carry on with other tasks.
Futures are based on the Future trait, which in its simplest form looks like this:
pub trait Future {
type Output;
fn poll(self: Pin<&mut Self>, cx: &mut Context) -> Poll<Self::Output>;
}
The Future trait is an asynchronous version of a Generator. It has a single method, poll, which is invoked by the executor to drive the future towards completion. The poll method checks whether the Future has finished its computation. If it has, it returns Poll::Ready(result). If it hasn't, it returns Poll::Pending and arranges for the current task to be notified when poll should be called again.
Async and Await
async and await are special syntax in Rust for working with Futures. You can think of async as a way to create a Future, and await as a way to consume a Future.
async is a keyword that you can put in front of a function, causing it to return a Future. Here's a simple async function:
async fn compute() -> i32 {
5
}
When you call compute, it will return a Future that, when driven to completion, will yield the value 5.
await is a way to suspend execution of the current function until a Future has completed. Here's an example:
async fn compute_and_double() -> i32 {
let value = compute().await;
value * 2
}
Here, compute().await will suspend the execution of compute_and_double until compute has finished running. Once compute has finished, its return value is used to resume the compute_and_double function.
While a function is suspended by await, the executor can run other Futures. This is how async programming in Rust achieves high concurrency: by running multiple tasks concurrently, and switching between them whenever a task is waiting on a slow operation, such as I/O.
Executors
An executor is responsible for driving a Future to completion. The Future describes what needs to happen, but it's the executor's job to make it happen. In other words, without an executor, Futures won't do anything.
Here's a simple example using the block_on executor from the futures crate:
use futures::executor::block_on;
async fn hello() -> String {
String::from("Hello, world!")
}
fn main() {
let future = hello();
let result = block_on(future);
println!("{}", result);
}
In this example, block_on takes a Future and blocks the current thread until the Future has completed. It then returns the Future's result.
There are many different executors available in Rust, each with different characteristics. Some, like tokio, are designed for building high-performance network services. Others, like async-std, provide a set of async utilities that feel like the standard library.
Remember that, as the developer, it is your responsibility to ensure that Futures are properly driven to completion by an executor. If a Future is dropped without being awaited or driven to completion, it won't have a chance to clean up after itself.
In summary, Rust's async/await syntax and Future trait provides a powerful model for writing asynchronous code. However, they come with their complexities and require a good understanding of the language's model of ownership and concurrency.
In conclusion,
Rust provides a powerful toolset for handling complex programming tasks, offering unparalleled control over system resources. It encompasses advanced types, traits, and async functionality, catering to both low-level and high-level programming needs. While Rust may seem initially daunting, the benefits it offers in terms of performance, control, and safety make the learning journey worthwhile. Understanding the concepts of ownership, borrowing, and lifetimes will be your guide as you navigate the intricacies of Rust. Embrace these principles, and you'll be well-equipped to tackle even the most challenging aspects of Rust programming. Happy coding!
Posted on June 20, 2023
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.
Related