Often, when facing scalability issues in our applications, the first reaction is to think about changing the language or framework.

The technologies we use are indeed VERY important for how fast and scalable our application can be. However, they are not the only factors to consider.

However, this decision may not be the best one, as there are several ways to make your application faster and more efficient without changing technology.

Before we dive into the details of how to scale your application, it's important to understand what happens when a request reaches the server. When this happens, a web server (such as NGINX or APACHE) is responsible for processing the request and forwarding it to the corresponding application.

In other words, the web server is the first point of contact between your application and the user.

But as we are talking about dynamic applications, the path taken by the request does not end at the web server. The code of your application needs to be executed in some way, either through a script (in the case of Laravel, for example) or a process managed by an application server (such as Passeger, in the case of Rails). And of course, the information manipulated by the application is stored in a database.

All this to say that the scalability of an application depends on many factors beyond technology choice. To ensure that your application is scalable and does not present bottlenecks, it is necessary to take into account all these elements and optimize them in the best possible way.

In your application workflow, everything can be synchronous, meaning that if your database takes one second to load, your request will be delayed by one second. In such cases, the choice of your framework won't make a difference, as the bottleneck is likely to be at the database, server, or some other point in your application.

If you want a fast app, you need to be fast at every process.

Identify the bottlenecks.

For that you can use some tools, like New Relic

There are situations where everything seems to be okay.

Your application is configured correctly, but still, everything seems a bit stiff and slow in your product. I will address 3 common scenarios:

1 - Slow tasks:

Imagine that you have a feature where when the user uploads a CSV file, your application scans each line of that archive and saves it in a database.

If your CSV has 15 lines, it will be quick, but if it has a million lines, it may start to get slow. In this case, there isn't much to escape 😔.

It's a naturally slow process and task. But a good solution to improve your user's experience is to implement the concept of QUEUES 😎.

A queue works like this: you take a process that is being executed synchronously, and you isolate it in what we call JOBS. So now we have a Job that we can call "ProcessCSV." You put this in a separate class, and when the user sends the CSV, instead of processing it at that time, you send that job to the queue, which is another process that will take the jobs sent, process them, and delete them after processing. In other words, you make your application more asynchronous.

But this requires good UX; you have to make it clear to your user. Your client needs to know what is happening with the task they requested for your application. For that, I also recommend reading about State Machines.

2 - Cache is love, but not always love is good.

Let's say you have an application that returns a common API. It makes a query and returns the data from a table. If the data doesn't have a very fast mutability, meaning if you know that in a little while, if the user doesn't make any changes, the data will be the same, you can work with a cache. When making a mutation in your application, it's possible to implement an update of your cache when the mutation is completed. If you want to add new data, you can manipulate your cache to add this new post instantly to your user, without needing a refetch every time the user changes something.

Do not have outdated data, be careful.

However, this is not always good to do. Let's say your site returns the updated value of Bitcoin in relation to the dollar. For this, it's not interesting to persist this data 100% in cache, considering that in a short period of time, your user wants to know the new value of Bitcoin, given that it fluctuates a lot.

You can watch a comprehensive video where I explain how I implement cache writing in my frontend application

Tools for cache and examples of using

Imagine that you need to build a news website. If your site is small, it's fine to make 30 requests and query your database 30 times. However, if your task is to create the primary news portal for your country, there will be millions of requests at any given time for data that typically doesn't change very often. After all, the most that will happen is a writer editing a few details or deleting the news entirely, but they won't typically make thousands of edits per day.

To handle this level of traffic, you can use tools such as Varnish HTTP Cache, which caches the information of a news article starting from the first user who accesses and makes the request. Once Varnish caches the page, subsequent users will receive a response that is saved in memory. This process allows you to avoid unnecessary synchronous requests and send a quick response to users.

In other words, Varnish intercepts the request and sends a pre-cached, beautiful, and fragrant response to your users without overwhelming the server with millions of unnecessary requests. However, it's important to take some security measures, such as validating whether the news has been altered or checking if the content has changed.

Using a tool like Varnish HTTP Cache is a great way to make your page faster and handle high traffic volume with ease.

Last but not least - IMPROVE YOUR CODE

I will write a entire article about that, but for now:

Improving your code is crucial for the success of any programming project.
Bad code make all feature more difficult and take more time to be deployed. And time is money.

If you're working in a team, it's important to have the right tools and processes in place to ensure that everything is running smoothly. While there are many ways to achieve this, here are some key tips that you can implement:

• Utilize efficient debugging tools to identify and resolve errors quickly.

• Thoroughly test your code to catch any potential issues before they cause problems.

• Use clear and descriptive variable names to improve readability and maintainability.

• Make use of Lint tools to identify and fix common coding mistakes.

... (add additional tips as appropriate)

By following these best practices and continually refining your approach, you can ensure that your code is well-organized, reliable, and easy to maintain, ultimately leading to more successful projects and happier stakeholders.

Thanks @sibelius for give the question.
Thanks Fabio Akita and Mateus Guimarães for explain me some topics.

Hope this post can be helpful!
For some feedback or more content, follow me on twitter or github