Scaling and performance tuning

Kubernetes provides the features to achieve rapid elasticity on the managed clusters. Service Mesh Manager and Istio rely on the features provided by Kubernetes to support differently-sized deployments ranging from a few hundred Pods and Services to supporting clusters loaded with thousands of them.

When it comes to modern (microservice-based) systems there are two challenges regarding the performance optimization:

  • First, it’s hard or even impossible to tell what are the best numbers and settings for a given deployment. This is due to the fact that you are no longer controlling the underlying hardware: any assumption on how many CPUs a workload requires is a speculation due to the various instruction per clock (IPC) and performance characteristics of modern CPUs. The same is true for Memory requirements: in case of Istio or Service Mesh Manager, the memory usage depends on the number of running Pods and the number of alerts, and due to rapid elasticity, it will most likely change during the lifecycle of the deployed solution or cluster.

  • The other challenge comes with the microservice architecture itself: the end user measured the performance of a given solution by experiencing the speed at which an HTTP call is served. Given that, in the background, multiple services are talking to each other to serve a single request, you do not just need to understand the performance characteristics of each service, but also the end-to-end call flow that resulted in the request being served.

Demo application with failure signals Demo application with failure signals

Service Mesh Manager

To address the challenges described in the previous section, Service Mesh Manager provides:

  • the necessary control to set the right performance parameters for Istio
  • the necessary tools to understand how a single workload behaves
  • the necessary tools (tracing and traffic taps) to examine end-to-end flows real-time

This guide covers all aspects of performance tuning, starting from generic workload performance tuning to how to fine-tune Service Mesh Manager to best suit the given deployment’s performance characteristics: