For low-traffic applications, Cloud Run is dramatically cheaper than App Engine.

Abstract

I was hosting a small web app as a side-project and looking to spend less money. I started out using Heroku, then moved to Google’s Cloud Platform. Using rigorous methods and markdown tables, I performed a science-inspired “how much does this cost?” comparison between App Engine and Cloud Run. This study finds that Cloud Run is usually the best option, although if you have money to burn are a “price insensitive consumer,” then App Engine is a bit zippier.

Introduction

Imagine you have a side-project-type web app and you’re looking to host it on Google’s Cloud Platform (GCP) but you don’t want to spend too much ca$h. Which GCP service do 4 out of 5 scientists recommend? Let’s find out.

Background

My incredible journey went basically thus: I built a small express app for upcoming Canadian holidays and wanted cheap but usable hosting. Initially, I was using Heroku’s $7/month Hobby Plan because at the end of the day month, it’s only $7. (ie, that’s like 3 coffees: ‘a coffee’ being the base unit of diminutive purchases.)

Heroku was really easy to get going with, to integrate with GitHub Actions, and to ssh into when I needed to fiddle with something. But around month five, it dawned on me that it was going to cost $7/month for the rest of my life, so I started looking for other options.

Pivoting to Google Cloud Platform (GCP)

GCP was the cloud vendor with the most bonus cash on sign-up, so I figured that was a pretty neutral and unbiased reason to pick it. However, as a hapless first-time user, there are a lot of “solutions” to choose from.

It seems like you’re not a real cloud vendor unless you can bury newcomers under an avalanche of vaguely differentiated products with abstract geometrical logos, so a straightforward question like “where do I host a basic express app?” didn’t have an obvious answer.

Cutting through the media bias with facts and logic, I was able to narrow it down by following the research methodology of googling “google cloud how do I host express app”.

The two options that popped up were:

1. App Engine
2. Cloud Run

Both services will run apps and I had an app to run. Seemed perfect: they anticipated me like how I anticipated Canadians are looking for information about holidays.

Methodology

By signing up, I was granted 300 (!!) GCP bucks, and as a long-time government employee I knew this meant I had to find a creative way to spend it before the end of the fiscal year. Are you thinking what I’m thinking? Let’s run a research study!

(This is where the science comes in.)

My research question was “Should I use App Engine or Cloud Run to host my fun but unprofitable app?”, and to investigate that I opted for the immersion method where I would assume the role of a developer trying to host an app on Google Cloud.

Setup

As a precursor, I needed to set up my app on both services simultaneously. For the initial setup, I used the Quickstart material provided by Google at no cost to embedded researchers like me. (Both Quicks-start are pretty easy to follow once you have the gcloud command-line tools installed.)

Overview: App Engine (AE)

• Node.js Quickstart for App Engine

On AE, my express app runs as a node process, like booting it up with npm start locally. AE is a traditional hosting platform: it runs continuously and serves requests as they come in. At the end of the month, you pay for the amount of time it was running, which is typically “the entire month”.

Overview: Cloud Run

• “Build and Deploy” Quickstart for Cloud Run

Cloud Run runs containers, so for each release you have to build a container and push it to GCP. Unlike App Engine, Cloud Run only runs when requests come in, so you don’t pay for time spent idling.

Containerized apps are more portable but not always something you focus on during development. It’s worth noting that the Cloud Run Quickstart provides 9 example Dockerfiles depending on your language of choice. (I used the Node.js one as a basis.)

Simulating traffic

At this point in the study, I had 2 instances of my app running:

1. In App Engine: https://hols-ae.nn.r.appspot.com/
2. In Cloud Run: https://hols-hzlcxvebra-ue.a.run.app/

Because real applications have real traffic, I set up a ping service to send requests to each site exactly once every 47 minutes for the rest of time, just like how a Real Human Being™️ would browse.

Having completed my setup, it was time to let the experiment run its course, so I passed the time doing highly academic things like rinsing noobs at dominion.games.

Duration

2 months.

Findings

There were 2 principal findings of the study.

1. For a low-traffic application, Cloud Run is dramatically cheaper than App Engine
2. App Engine seems to respond slightly faster

1. Ongoing costs — Cloud Run wins ✅

Wow.

App Engine runs 24/7 for the entire month whereas Cloud Run only runs when serving requests, and the difference is startling.

Previously, I had been paying $7 a month for Heroku’s Hobby Plan.

• App Engine would cost me about 50% more
• Cloud Run costs 99% less, oh my goodness

So basically it’s a blowout win for Cloud Run here.

2. Request latency — App Engine (usually) wins ✅

I also used some online speed test tools to measure the response times of my 2 instances. The results weren’t totally consistent, but App Engine generally responded more quickly.

Pingdom Speed test

(Results of 3 runs from São Paulo)

Here we see App Engine responding on average 56 ms faster than Cloud Run (although in 1 case, Cloud Run was faster). The huge caveat here is that these times vary widely between runs, sometimes tripling or quadrupling depending on Who The F*ck Knows.

WebPageTest

(Results of 3 runs using “3G” download speed.)

Again, keep in mind that these numbers shift around between runs.

Why is App Engine faster?

This isn’t totally clear to me, but I can speculate.

The one measurable difference I noticed is that that the total request size from Cloud Run was larger because it doesn’t gzip files by default.

The Pingdom Speed Test for Cloud Run recommended I Compress components with gzip, and looking through the requests, my combined .js assets are indeed about 6 KB larger.

Downloading bigger files makes your site slower, but I don’t think that’s the whole story.

The big difference between the two services is that Cloud Run doesn’t run your container unless it’s getting requests. When a request comes in, it does 3 things:

1. boots up the container
2. serves the request
3. shuts down the container

It seems likely that the extra time needed to boot up the container adds to the total request time, leading to an average slower response time from Cloud Run.

Of course, you also save a lot of money doing it this way, so the tradeoff here is whether you care more about optimizing your speed or your cost.

Findings

For me, the findings are decisive. If you’re a hobbyist developer and you want to host your fun app for next-to-free, you should definitely use Google Cloud Run.

However, if money is no object, then you can pay exponentially more per month for a marginal speed boost on App Engine.

Further reading

• Read more about why Google Cloud Run is better than other hosting options
• For an excellent intro to Docker, check out this excellent guide by Robert Cooper
• Check out Google’s “Build and Deploy” Quickstart for Cloud Run
• Use Github Actions to deploy automatically to Cloud Run