Introduction

A team of talented individuals embarks on a journey to build something cool for the rest of their colleagues to help them in day to day life at the office. This bright idea of a super powerful, social-network-like app for company employees emerged. User flows are quickly sketched out, dozens of features planned, technical diagrams are drawn, we had it all figured out and were super motivated to start. On top of that, we'll build it and host it ourselves on some VMs on premise!

You may have guessed - it didn't see the light of day.

In this blog post, I'll share why this approach failed and how, in a second attempt and by adopting AWS and Serverless, we successfully developed and deployed a functioning MVP on a limited time and budget.

The First Attempt: A Lesson in Complexity

We have our architecture laid out and it's time to make those diagrams a reality. We're going to deploy Kubernetes to our existing servers with Apache Kafka as the backbone for the system. The app is going to be packed with features so it needs to be powerful, resilient and future-proof and what not.
Setting up the Kubernetes cluster required significant effort, even with the help of our seasoned ops colleagues, but eventually it was up and running on 5 nodes, managed in house. Next, configure Kafka and integrate it with several microservices that are being developed in the meantime - also not a walk in the park. Neither was setting up self-managed GitLab Runners for deploying the microservices.

I could go on with a few more things, but I hope you see where this is going. Not a single feature was really finished! And when I say finished, I don't mean works perfectly or has no bugs, I mean just deployed and working OK. And we had a team with plenty of expertise, so the reason definitely wasn't lack of talent.

Managing all the infrastructure components took a big bite of our time and diverted our attention from building core features of our product.

Soon enough, people had to focus on different priorities within the company, so the team disassembled and the project was abandoned.

A New Approach: Embracing Serverless and AWS

Some (a lot of) time passed, our company adopted the cloud, everybody gained more experience with it along the way, especially on AWS. Coincidentally, a part of the original team was working together again and, determined to learn from our past mistakes, our idea was about to get a fresh start. The goal now is to get a usable app in the hands of first users as soon as possible. After a few UI and cloud architecture sketches, we were ready to start. We took out one feature from the original, full-blown app idea, and worked out an MVP scope for it. It was basically a web application communicating with a REST API and a WebSocket to support real time group chat. Laser focused on our goal, relying heavily on AWS managed services, the first deployment happened within a week or so. To give you an idea of our starting landscape, take a look at the diagram below.

One piece is missing and that's IoT Core, but I just wanted to use this new set of AWS icons for Excalidraw so badly.

Trimming Down Features: The Importance of Scope Control

While cloud and Serverless enabled us to be productive already in the first week, they will not protect you from scope-creep! Everybody from the team had brilliant ideas on what to implement next. Arguing how "adding just this small thing would be great for this and that", or slipping into a trap of "this will take just 2 additional hours and bring more value". Fiercely cutting down on scope expansion was instrumental in keeping the team focused on the main goal - and that is serving the first group of users as soon as possible.

This was especially important as we knew our time to work on this project is quite limited - new assignments in the company were looming around the corner.

User Feedback Or Where Are The Emojis?

We are building something for our colleagues, for internal use, basically solving our own problems, and that gives us good insight in which features we need to build first, right? Surprisingly, the feedback from every single person from our first group of pilot users was that the chat needed to be richer, more precisely to support emojis and emoji reactions on messages. On the other hand, people are really used to rich chat features, since they have them in every actual chat app. Even if it wasn't considered crucial from our point of view, we realized that missing this could lower the chances of anyone using our app. It was the first thing on our backlog the following week.

So, we missed an important feature, but the fast feedback enabled by our cloud powered development setup makes it easy for us to quickly iterate and add the missing value.

Next Steps: Preparing for Production

We are still working on a few quick wins that our first group of users suggested, while aiming to tighten a few more knots before we open the app to all users. Currently, it's deployable and we believe it is functionally OK, so next step is validating that it's secure, reliable and performant.
Security will be checked by penetration testing and to make sure the app is reliable and that we spot any performance bottlenecks quickly, we've setup CloudWatch monitoring dashboards and alarms. Having an overview of metrics like number of API requests, latency, error count, and alarms on any server error enables us to react pro-actively. This would've been much harder if we were running this on-prem.

Conclusion

This story is an example of how AWS and its Serverless services can save time and resources when building under constraints. Apart from that, it touches the subject of scope-creep and the importance of keeping it under control when building an MVP. Also, discusses the steps to take after, like ramping up security and considering ways to measure the general quality and success of the product as it evolves.

Not to get ahead of ourselves - we still have some work to do before we can think about measuring if the product is successful. However, for things like basic monitoring, alarming and automation - AWS makes this ridiculously easy to do that it's a no-brainer.