Should I put my service in the private subnet or the public one?

This age old question has been around since almost the beginning of AWS. And it seems like a simple question with a simple answer. However, the real truth is there is no answer.

Private Subnets

Before we get to it, let's review what is even a private subnet. AWS allows you to create virtual networks, these networks are called VPCs. In each VPC you can isolate compute resources. The logical isolation inside a VPC is known as a subnet. Subnets can connect to internet and the internet can be connected to them.

Okay, so where does the private part come in. There is actually no such thing as a private subnet in AWS, or public for that matter. It's a completely made up term that doesn't exist in AWS, because there is only "a subnet". However, frequently the term is associated with the following criteria:

• Has publicly IP addresses
• Internet can be routed the IP addresses
• The IP addresses can be routed to the internet

If you are missing any of those then technically it isn't a public subnet, does that make it a private one? Who knows!

The Public Subnet Risk

Now that we know what a private subnet is let's talk about the risk of using a public one for your product/service/application/technology.

What risks are associated with using a public subnet?

The problem with this question is that risk is not binary. It could be concluded that there is more of a risk that something bad will happen than if it is in a private subnet.

Is that true though?

The VPC and Subnets offer each their own security appliances to protect resources, and the only difference is internet connectivity.

Which means the question becomes:

Is a subnet connected to the internet a risk?

Maybe?

See, just because there is a route to your subnet compute resources doesn't mean that you've opened the firewall of the instance or the Security Group to allow connections to the machine. But you could.

The question isn't "Is my instance or database directly connected to the internet a risk?" Because that has a simple answer: YES

Having your subnet connected to the internet doesn't offer attackers any additional means to get to your compute resources. And further, often in the case of using a private subnet you've added a NAT and internet gateway connecting through your public subnet. So in essence you've reconnected them to internet, albeit a different way.

Actually the truth here is if your technology is sufficiently protected it doesn't matter what setup you use.

What does Sufficient protection look like?

The problem with this conclusion is that we can't list out every possible attack vector because there are an infinite number of them. And further, our technology isn't deployed in isolation. We deploy new software packages, application versions, fixes, and we also may change the infrastructure of the VPC over time. That means we can't just evaluate the RISK at a static point in time.

So the risks are actually whatever you conceive to be a risk at a certain point in time. No one can tell you what your risks are. They can tell you what their risks are, and then point you to how they solved them, but even then it won't include everything.

Assume you get a list of risks, We know that isn't a full list, because the list of risks is infinite. And based on that list you go with a public subnet. One day will you regret when something happens anyway, maybe? "If only I had went with a private subnet..." What happens if you learn about a new risk you aren't willing to deal with, will you switch back to a private subnet "before something happens..."

Further, you need to understand that just because there is a risk doesn't mean it should be considered. It is risky to go outside because you might get hit by a car. Cars are dangerous. But that could be a risk you ignore.

1. Is it a risk, yes
2. When is it a risk, when outside
3. Do you care, no (at least no in this case)

So what do we do?

Well we know that we can't have a full list of the risks, and we know that we might not even care about the risks on the list, but the most dangerous thing is leaving the conversation thinking you have all the risks because someone on the internet told you a list. You might be missing the most critical aspect that wasn't on the list.

That means you need to develop the list yourself, what are your risks?

This is known as threat modeling. You come up with actual risks that you care about, than then you evaluate whether the solution you have is the right one.

Threat Modeling

It may seem like it isn't important to do this, but I'll explain why it is. Let's look at an example.

We put our instances in the private subnet, but then we realize that we need to add an ALB to grant access from the internet to your service AND we also realize that we need to grant access from your service to the internet.

The reasons are usually something like, well this is a product users need to access it, and it needs to get updates or make calls to other services to perform its functions.

As part of making this changes, likely in terraform, you update the security groups attached to the instance. And in doing so, you accidentally remove access to the database. This accident can easily be happen since database access is protected by a security group and configured in terraform, therefore is something you can break by changing infrastructure.

This is an example of: Using a private subnet increases risk.

A common issue is one of the form that increased complications increase risk. I'm going to repeat that again:

Complications = Risk

The more complicated your systems are, then the more likely you will be to accidentally do something wrong. At the same time, the less safeguards you have in place the more likely you are to accidentally do something wrong.

So in reality we don't know if our infrastructure choices have increased or decreased risk. Each change can increase or decrease the risk, and we won't know which way it goes.

The Risks

That's because risk is a spectrum, how risky something is can increase or decrease by changes to the circumstance. We don't have perfect knowledge of the circumstance as well as the perfect knowledge of all the risks.

The reason for this is because, there are an infinite number of risks, and an infinite subset of the risks change by altering the aspects of the situation.

When switching from a public subnet to a private subnet, some of the risks decrease in riskiness and others actually increase, as we have seen. That means that a private subnet is both more secure and less secure at the same time. And Security isn't the only kind of risk that we can talk about.

So the only valuable conversation to have is to talk about what risks do you care about, and then decide if those risks are higher or lower with a private subnet.

Conclusion

There was recently a great talk about our collective inability to Measure Security Outcomes.

I bring up this topic as it is one that frequently occurs and seams to not have any concrete and satisfiable answer yet. It's also incredibly relevant for my teams in building Authress, which focuses on providing user identity and access control. We care about making Authress as secure as possible to protect our customers' users. So you can imagine questions about security are always present in our conversations.

And if you are interested in this topic and others, you can join our Security Community