Scaling Out versus Scaling Up

https://www.flickr.com/photos/torkildr/3462607995
Torkild Retvedt (flickr.com)

So, cloud computing.  Kind of a big topic with so many different things that can be discussed.  But’s let’s focus on one very narrow thing:  scaling up vs. scaling out.

There are two main ways of increasing the performance/capacity of an application and that is by scaling an application up (faster processors, bigger machines) or scaling it out (more machines).  When you have an application on-premise, in your own data centre, you get your choice of how to scale an application.

We have some applications that are designed to be scaled up.  They are designed in such a way that the only way to improve either the response time or the capacity is to increase the speed of the processors in addition to adding processors.  To be brutally honest, many of our applications are built this way.   Certain processes, certain features (batch processing anyone?) are written in a single-threaded manner and the only way to make it faster is to give it faster processors.  And sometimes it is not necessarily a complex process, just going from one page to another in the application.  The problem is that there is so much processing going on from page to page that no matter how you slice it, a faster response is only available if you get faster processors.

This isn’t going to happen.  Yes, there are advances in processor technology – smaller die size, optimized micro-code, better path detection, larger L1/L2 caches, increased bus speed – that increase the overall responsiveness, but there is a limit.  To truly improve the performance you need to do a couple of things:  implement multi-threaded applications or reduce the amount of processing required to go from page to page.

Applications that can scale out aren’t excessively complex and are on the lower end of processing power required to provide a response.  Increasing the processor speed will cut response time,  but response time isn’t that long unless there are a lot of requests in the queue.  Better overall throughput is achieved by adding more machines to do the processing.  This is scaling out.  In the example above, where you reduced the amount of processing required to go from page to page, you are turning the application from one that requires scaling up to one that works scaling out.

The cloud works well with scaling out, not so well with scaling up.  Cloud infrastructure components are designed to be interchangeable with each other.  Few cloud providers offer “souped up” processors.  They offer the standard “processor”.  Scaling out works better.  Indeed many cloud providers offer the capability where if your website is busy (CPU > x%) it can spawn another copy of your website.  So, instead of having one website you now have two.  If they get busy it spawns another.  And another.  And another, until the websites are no longer busy.  If the websites go idle (CPU < y%) then it turns one off until, eventually, you’re back to a single website.  That is the power of scaling out in the cloud.  Granted, you need to design your application properly in the first place, but think of the power and flexibility this provides.

Scaling up is limited.  Processors only run so fast and no faster.  The ability to get “faster” processors is limited to technological changes.  Scaling out?  That’s the easy part.  Even if you don’t architect for the cloud you can still clone a virtual machine to get another one.  It’s not as flexible, but still better than waiting for Intel to announce a new set of processors.

Leave a Reply