Hi everyone,

I know this is a bit far fetched but just wanted to see what other developers have done in the past.

So let’s imagine we have 3 sets of data that are about 300 million rows in each set, ie 3x MySQL tables and each of them hold 300 million rows and let’s say 20 columns of mixed data (varchar and integers).

What’s the best way in theory to collate data from each set and return it to the user under a second? How is it done?

Horizontal scaling?

Hardware and power or is the bigger part really to do with data design and the way you go about retrieving the data?

I would like to know other developers experiences similar to the above and how they overcame any hurdles along the way.

Thanks everyone :)