Some months ago, I was at the AWS re-cap event in Madrid to view all the Highlights from the re:Invent 2022 and the presentation of the new region of AWS Spain in Zaragoza.

In the Q&A session, I asked a question regarding latencies.

Before deeping dive, what is latency?

Latency is a term used to describe a time delay in a transmission medium.  In free space, light travels at 299,792,458 meters per second.  This equates to 299.792 meters per microsecond (µs) or 3.34µs per kilometer.

I wanted to know what the latency is for a Spanish customer using Direct Connect to get transported from the DX locations located in Madrid (located either in Interxion or Equinix) to Zaragoza, to the new cluster in Spain.

Considering a straight line, Zaragoza is around 320 km away from Madrid. It is not in the same metropolitan area

What I asked is a tricky question, because it's not just about the distance of the various fiber routes with the travel speed of light, but also the number of network devices the data stream must pass through to the cloud of AWS Spain.

I got as an answer an estimation of less than 10ms.

After many months, I decided to resolve this equation and do this measurement; we are engineers and like precision ;). I will try to keep the solution simple; let´s get started:

To perform this measurement, I created the following infrastructure in AWS:

The idea behind this infrastructure is to access my Private Host 1 in the private subnet via SSH from the bastion host, which is available from the internet.

Looks easy, right?

Imagine replicating the same infrastructure using a different IP space but keeping the same configuration.

Now that we have these two infrastructures in place, how can we interconnect them for performing this measurement?

Using the Direct connect Service (DX), with the help of an interconnection partner. This is the list of partners; for this lab, I will use DE-CIX.

We will interconnect the VPC1 with a DX connection to a location where AWS and the partner are present → Interxion MAD2, in Madrid.

We will do the same with the VPC2 using another DX connection ending in the same DC.

Once they are interconnected, we will ping from Private Host 1, to Private Host 2 for measuring the path until Madrid from Zaragoza.

Maybe you will be asking, how will you interconnect one DXcon with the other DXcon? Do you have a rack there for routing and doing measurements?

The answer to the last question is yes and no; we will use the DE-CIX CloudROUTER to interconnect the two AWS Direct Connections in Interxion.

This Cloud Router is a VRF (Virtual Routing & Forwarding) instance executed in carrier-grade networking gear. It extends where it is needed, in this case, to the Interxion MAD2 location where the partner and AWS are present.

Note: If we want to interconnect two instances in different VPCs, we use a VPC Peering connection in AWS.

For this exercise, because we want to measure the path until Madrid, we do this architecture using the DE-CIX services.

If we send a ping from Private Host 1 to Private Host 2, the ping flows from Zaragoza to Madrid and from Madrid to Zaragoza, ending in Private Host 2 and returning to Private Host 1.

Because we want to get the latency of half path (from Zaragoza to Madrid only), we must divide it by two if we perform a ping.

Measurements

I´ve done two ping runs with different window sizes to perform the measurements. As you can see, the latency can vary a little bit. This depends on the route AWS is taking inside its backbone. It seems there are two different ways to reach the destiny, one with less latency than the other.

Also, because there are different availability zones (AZs), the latency can vary depending on where our resources are located.

Window size 128K – Run 1

Avg = 11.934 / 2 = 5.967 ms

Window size 128K – Run 2

Avg= 16.441 / 2 = 8.2205 ms

Window size 256K – Run 1

Avg = 11.928 / 2 = 5.964 ms

Window size 256K – Run 2

Avg = 16.451 / 2 = 8.2255 ms

Window size 512K – Run 1

Avg = 11.946 / 2 = 5.973 ms

Window size 512K – Run 2

Avg = 16.452 / 2 = 8.226 ms

Window size 1024K – Run 1

Avg = 11.975 / 2 = 5.9875 ms

Window size 1024K – Run 2

Avg = 16.492 / 2 = 8.246 ms

Average of the taken samples:
Run 1 ≈ 5.97 ms
Run 2 ≈ 8.23 ms

Conclusion

The answer to my question depends; it depends on which internal path of the AWS backbone you land and where your resources are; you will get either 6 or 8 ms (approx.) to reach the cloud from Madrid to Zaragoza.

Personal thoughts

This mini-lab was fun to do; having the opportunity to integrate cloud connections with this technology is a grand experiment that everyone should try.

In the future, I could prepare another lab for migrating from one cloud to AWS using high bandwidth DX Connections in combination with the Cloud Router and measuring the throughput. What do you think?

I read you in the comments :D