Hello! Before we start our little tangent, I do have a caveat. TLS and security is important! Our data is flying everywhere across the internet and TLS (also known as SSL) is what keeps it safe.

This chapter is completely optional, and it may be better to move on to chapter 3, get some context and then come back to read this. This chapter really exists because the rustls crate doesn't have working examples and I wanted to have a place I could refer back to, to get working snippets to play with.

https://docs.rs/rustls/0.18.1/rustls/

I am going to outline 4 examples in total. The first example will be my own explanation and code of the TLS process just to make it clear what is happening when we set up TLS.

The next 2 examples will be working examples from the rustls docs page.

The last example will be what we will ultimately be using in our Gemini client. This is where we will remove the verification of the TLS certificates.

Feel free to try out the examples in a new project.

cargo new tls-test

1. The Flow Example

Before we get started, we need to include some crates.

./Cargo.toml

[dependencies]
rustls = version="0.18"
webpki = "0.21"
webpki-roots = "0.20"

Now that we have everything we need, we can get started on the code.

./main.rs

use std::sync::Arc;
use std::net::TcpStream;
use std::io::{Write, Read};
use rustls::Session;

fn main() {
    let mut config = rustls::ClientConfig::new();
    config.root_store.add_server_trust_anchors(&webpki_roots::TLS_SERVER_ROOTS);

    let rc_config = Arc::new(config);
    let google = webpki::DNSNameRef::try_from_ascii_str("google.ca").unwrap();

    let mut client = rustls::ClientSession::new(&rc_config, google);

    let request = b"GET / HTTP/1.1\r\nHost: google.ca\r\nConnection: close\r\n\r\n";

    let mut socket = TcpStream::connect("google.ca:443").unwrap();

    println!("1. Request TLS Session");
    client.write_tls(&mut socket).unwrap();

    println!("2. Received Server Certificate");
    client.read_tls(&mut socket).unwrap();

    println!("3. Check certificate");
    client.process_new_packets().unwrap();

    println!("4. Write out request");
    client.write(request).unwrap();

    println!("5. Encrypt request and flush");
    client.write_tls(&mut socket).unwrap();

    println!("6. Decrypt response");
    client.read_tls(&mut socket).unwrap();

    println!("7. Check certificate");
    client.process_new_packets().unwrap();

    println!("8. Read data");
    let mut data = Vec::new();
    client.read_to_end(&mut data).unwrap();

    println!("9. Data");
    println!("{}", String::from_utf8_lossy(&data));
}

The first line sets up our configuration object that will act as our verification.

In this configuration object we will add out root certificates, in this case it comes from crate webpki_roots. This is a certificate store generated from Firefox.

Next, we make sure our url, "google.ca" is valid and create a DNSNameRef for it.

We then setup a ClientSession object.

We construct a valid HTTP request. Host and Connection are mandatory in this case as we do need the server to close the connection for us to know when a response is really done. We don't want to calculate the content length and such information!

Next we set up a socket.

At this point we have 2 things in our little example, we have a TLS session object and we have our socket that we write to. The next part will muddy the two together!

The first step is to trigger TLS, the client needs to connect to the server on port 443. This is what the first write_tls call does.

The second step we need to do is read_tls, we need to read the server's TLS certificate.

process_new_packets will panic if the certificate for some reason is incorrect or doesn't match what's in the certificate store, webpk_roots. This function is what verifies our TLS connection.

Now that we have verified the server's certificate, we can write out the request we constructed before.

We then flush this by doing another call to write_tls. This is where the request would get encrypted and sent to the server.

Next we call read_tls as the response the server sends to us will be encrypted.

We call process_new_packets again this will decrypt the data.

Now we can read the data from the TLS session into a vector.

We then convert the vector of u8s into a string and voila! We have used TLS to make a request and got the response!

The next example is very much the same as this example except it uses a loop to get around having this meaning separate read and writes.

The other thing to note is that we use our TLS session (our client object) to do all of our write and reads, the socket exists but we don't interact with it directly because we need the TLS session to encrypt and decrypt the data.

In the third example we will see the combining of the 2, client and socket behind one interface that we can use much more easily. But before we get there, let's look at the rustls main example.

2. The rustls Example

./main.rs

use std::sync::Arc;
use std::net::TcpStream;
use std::io::{Write, Read};
use rustls::Session;

fn main() {
    let mut config = rustls::ClientConfig::new();
    config.root_store.add_server_trust_anchors(&webpki_roots::TLS_SERVER_ROOTS);

    let rc_config = Arc::new(config);
    let google = webpki::DNSNameRef::try_from_ascii_str("google.ca").unwrap();

    let mut client = rustls::ClientSession::new(&rc_config, google);


    let request = b"GET / HTTP/1.1\r\nHost: google.ca\r\nConnection: close\r\n\r\n";

    let mut socket = TcpStream::connect("google.ca:443").unwrap();

    client.write(request).unwrap();

    loop {
        if client.wants_write() {
            client.write_tls(&mut socket).unwrap();
        }
        if client.wants_read() {
            client.read_tls(&mut socket).unwrap();
            client.process_new_packets().unwrap();

            let mut data = Vec::new();
            client.read_to_end(&mut data).unwrap();
            if data.len() != 0 {
                println!("{}", String::from_utf8_lossy(&data));
                break;
            }
        }

    }
}

This is the example from the rustls docs page. I made one change here which is that I break the loop after we get one piece of data from the server, otherwise we'd be in an infinite loop in a connection that had already ended.

The break isn't needed had we kept the connection type as something like keep-alive.

The loop removes the need for the explicit labeling of steps that the flow example had!

3. The Stream Example

./main.rs

use std::sync::Arc;
use std::net::TcpStream;
use std::io::{Write, Read};

fn main() {
    let mut config = rustls::ClientConfig::new();
    config.root_store.add_server_trust_anchors(&webpki_roots::TLS_SERVER_ROOTS);

    let rc_config = Arc::new(config);
    let google = webpki::DNSNameRef::try_from_ascii_str("google.ca").unwrap();

    let mut client = rustls::ClientSession::new(&rc_config, google);
    let mut socket = TcpStream::connect("google.ca:443").unwrap();

    let mut stream = rustls::Stream::new(&mut client, &mut socket);

    let request = b"GET / HTTP/1.1\r\nHost: google.ca\r\nConnection: close\r\n\r\n";

    stream.write(request).unwrap();
    let mut data = Vec::new();
    stream.read_to_end(&mut data).unwrap();
    std::io::stdout().write_all(&data).unwrap();

    println!("{:?}",String::from_utf8_lossy(&data));

}

This example uses the Stream option from rustls which marries our socket and our client together and gives us one easy interface to use!

Much better!

4. TLS but No TLS!

Finally let's look at removing TLS verification completely. We'll have our connection and data go over TLS but won't validate anything.

This is dangerous and this feature is gated behind both a crate feature flag and is also gated with in the code itself. Let's start enabling this!

To enable dangerous mode for our TLS we need to update our Cargo.toml.

./Cargo.toml

[dependencies]
rustls = { version="0.18", features=["dangerous_configuration"] }
webpki = "0.21"

Now that we have dangerous_configuration added to our Cargo.toml file we can begin working on our main function.

./main.rs

use std::sync::Arc;
use std::net::TcpStream;
use std::io::{Write, Read};
use rustls::{RootCertStore, Certificate, ServerCertVerified, TLSError, ServerCertVerifier};
use webpki::{DNSNameRef};

struct DummyVerifier { }

impl DummyVerifier {
    fn new() -> Self {
        DummyVerifier { }
    }
}

impl ServerCertVerifier for DummyVerifier {
    fn verify_server_cert(
        &self,
        _: &RootCertStore,
        _: &[Certificate],
        _: DNSNameRef,
        _: &[u8]
    ) -> Result<ServerCertVerified, TLSError> {
        return Ok(ServerCertVerified::assertion()); 
    }
}

fn main() {
    let mut cfg = rustls::ClientConfig::new();
    let mut config = rustls::DangerousClientConfig {cfg: &mut cfg};
    let dummy_verifier = Arc::new(DummyVerifier::new());
    config.set_certificate_verifier(dummy_verifier);

    let rc_config = Arc::new(cfg);
    let google = webpki::DNSNameRef::try_from_ascii_str("google.ca").unwrap();

    let mut client = rustls::ClientSession::new(&rc_config, google);

    let request = b"GET / HTTP/1.1\r\nHost: google.ca\r\nConnection: close\r\n\r\n";

    let mut socket = TcpStream::connect("google.ca:443").unwrap();

    let mut stream = rustls::Stream::new(&mut client, &mut socket);

    stream.write(request).unwrap();

    let mut data = Vec::new();
    stream.read_to_end(&mut data).unwrap();
    println!("{}",String::from_utf8_lossy(&data));

}

Our original ClientConfig object used the certificate store from webpki_roots, now that we are removing that dependency, we need to replace it with something. In this case we're going to write our certificate validator.

This is where the DangerousClientConfig comes in. We use this struct to mutate our cfg object so that we can change the default certificate verifier.

We do this by first creating a struct, DummyVerifier which will contain nothing but an override for ServerCertVerifier. We will write this trait to do nothing but simply accept the certificate. We then use set_certificate_verifier to change the certificate verifier on our config object.

With that! We have our TLS session now using the certificate verifier that we wrote. In this case it will simply say that every certificate is valid. Which is pretty dangerous but we're going to be needing it for our Gemini client.

The Gemini spec allows the client to do whatever it wants with the certificate so it is valid to just ignore it but the recommendation is to do TOFU pinning. We may end up implementing this in a future chapter but because we are building a very basic client, for now just being able to ignore the server's certificate is enough.

Gemini & rustls

This next part is very flukey, I see what's going on but I don't understand what's going on. This unfortunately is also the code we're going to be using in our client which is a bit frustrating.

There is an updated version that I'm much happier with in Chapter 4 - The Visit Function but I used this as the base for it so I'll leave this as is. You should be able to adapt the below code and use the improvements in the later chapter together.

use std::sync::Arc;
use std::net::TcpStream;
use std::io::{Write, Read};
use rustls::Session;
use rustls::{RootCertStore, Certificate, ServerCertVerified, TLSError, ServerCertVerifier};
use webpki::{DNSNameRef};

struct DummyVerifier { }

impl DummyVerifier {
    fn new() -> Self {
        DummyVerifier { }
    }
}

impl ServerCertVerifier for DummyVerifier {
    fn verify_server_cert(
        &self,
        _: &RootCertStore,
        _: &[Certificate],
        _: DNSNameRef,
        _: &[u8]
    ) -> Result<ServerCertVerified, TLSError> {
        return Ok(ServerCertVerified::assertion()); 
    }
}

fn main() {
    let mut cfg = rustls::ClientConfig::new();
    let mut config = rustls::DangerousClientConfig {cfg: &mut cfg};
    let dummy_verifier = Arc::new(DummyVerifier::new());
    config.set_certificate_verifier(dummy_verifier);

    let rc_config = Arc::new(cfg);
    let google = webpki::DNSNameRef::try_from_ascii_str("gemini.circumlunar.space").unwrap();

    let mut client = rustls::ClientSession::new(&rc_config, google);

    let request = b"gemini://gemini.circumlunar.space/servers/\r\n";

    let mut socket = TcpStream::connect("gemini.circumlunar.space:1965").unwrap();

    println!("1. Request TLS Session");
    client.write_tls(&mut socket).unwrap();

    println!("2. Received Server Certificate");
    client.read_tls(&mut socket).unwrap();

    println!("3. Check certificate");
    client.process_new_packets().unwrap();

    println!("4. Write out request");
    client.write(request).unwrap();

    println!("5. Encrypt request and flush");
    client.write_tls(&mut socket).unwrap();

    println!("6. Decrypt response");

    loop {
        while client.wants_read() {
            client.read_tls(&mut socket).unwrap();
            client.process_new_packets().unwrap();
        }
        let mut data = Vec::new();
        let _ = client.read_to_end(&mut data);
        println!("{}", String::from_utf8_lossy(&data));
    }
}

From what I can tell, the interaction of the server and rustls is a bit wonky. rustls doesn't seem to be waiting for responses from the server and so we need to be sitting inside a loop checking for data. This is why we let read_to_end come back with nothhing, if it does, we simply continue with our looping.

Something noteworthy here is that when we do go to use this logic, we'll need to somehow know when to break our loop but let's deal with that when we get there.

Hopefully these examples can shed some light on how rustls works and I may in the future sit down and see if I can understand what's going on in with the sockets. Likely the socket interface itself is non-blocking and it's confusing me but I had thought it was blocking.

Alright! Our little side adventure is over and now we can actually start working on our code!