Building systems for scale

From the archive: originally published on the Eastpoint blog on 18 February 2019. Some details reflect that time.

As more people get access to the internet and smart devices, the audience for technology services is growing. A big concern for both start-ups and established enterprises is whether systems can scale to demand.

At Eastpoint, we work with multiple clients building systems with the ability to scale based on demand, and I’d like to share some of the scaling patterns that are required to build large distributed systems. The term scaling pattern refers to the repeatable processes that can be used on multiple applications that need to handle large variations in the number of users, or other workload.

I’m going to give a quick glimpse of how we helped one of our recent clients build their next generation hybrid cloud system.

Synchronising mobile and cloud data at scale

To realise our client’s objectives, we built two mobile apps (iOS and Android) and several web-based systems. For the mobile applications we used Xamarin, a cross-platform framework, allowing us to share the core logic across both apps and help reduce delivery time. These mobile apps connect to web-based services, syncing data frequently.

The mobile app is a single interface for a single user but on the web apps, there will be millions of apps simultaneously trying to sync data. The complexity comes with these web systems, as the apps will be used across multiple countries and lots of users. The systems needed to be scalable and should in no way reduce the overall user experience on the mobile apps.

Scaling patterns to consider

Now coming to the topic of building systems to scale. We follow different scaling patterns, so that the systems are performance optimised for high traffic periods.

Below are some of these patterns:

1. Scaling up: This increases the overall hardware capacity to make sure the performance is not reduced, based on traffic.
2. Scaling out: Replication of the same application to multiple instances based on traffic. A load balancer is used to route traffic.
3. Microservices: This is a new approach that helps to build many small services that only serve one aspect of the overall functionality, allowing scaling of the specific service as required. Well-known services such as Google, Netflix and Office365 use this approach to scale their systems.
4. Downtime reduction: Having multiple environments to reduce downtime and achieve high availability. Also often referred to as redundancy.
5. Caching: Using a high availability and low throughput key store database like Redis for caching frequent used data. This approach helps us maintain a common cache across multiple instances of web applications that are hosted in different geographical locations.
6. Offload processing: Using background worker processes to help offload some load from the main web servers.
7. Cache invalidation: Caching helps to improve performance on the application but not validating caching at times may fill up the cache store and reduce performance.
8. Distributed storage: Using geo distributed storage like Azure Cosmos database to reduce network latency.

In addition to these patterns, we can also use a hybrid cloud approach, using services across Google Cloud Platform, Amazon Web Services and Microsoft Azure. The same service offerings by your preferred cloud vendors may not always best suit your requirements and going with a hybrid cloud approach solves this problem.

There are many more patterns other than the ones listed that can be used to scale systems horizontally and vertically. But I think those shown are the most important patterns to consider when building systems to scale.

We consider ourselves your technical partner and understand your passion towards your product. We can help you achieve planet-scale applications - let the journey begin!

Follow Eastpoint on Twitter