In today’s fast-paced cloud-native world, the speed at which your applications can start and scale is paramount. Faster pod startup times mean quicker responses to user demand, more efficient resource utilization, and a more agile development and deployment lifecycle overall. We’re continuously working to enhance the performance of Google Kubernetes Engine (GKE) to help you achieve these goals.
Previously we introduced container image streaming in GKE, a feature designed to significantly reduce image pull times and accelerate application startup. Today, we’re excited to announce a new set of performance improvements to GKE container image streaming.
These enhancements can help your GKE workloads start up faster and run more efficiently, particularly ones suffering from long startup times due to large container images. Specifically, AI/ML model serving applications will benefit from the improved startup times.
What’s new?
The performance boosts stem from a combination of targeted client-side innovations and ongoing optimizations to our image-streaming backend infrastructure.
A key improvement on the client-side is new intelligent read-ahead capabilities. These allow GKE to proactively fetch image data that is likely to be requested next, minimizing the time your applications spend waiting for data during startup. This works in concert with improvements to the image streaming backend, ensuring that your containers get the data they need, when they need it — just faster.
Alongside these client-side enhancements, we’ve made a number of improvements to our backend that help ensure that the image data is served efficiently and reliably, contributing to the overall speed and stability of the image streaming process.
- aside_block
- <ListValue: [StructValue([('title', '$300 in free credit to try Google Cloud containers and Kubernetes'), ('body', <wagtail.rich_text.RichText object at 0x3e6e300571c0>), ('btn_text', 'Start building for free'), ('href', 'http://console.cloud.google.com/freetrial?redirectpath=/marketplace/product/google/container.googleapis.com'), ('image', None)])]>
Measuring the gains: Our benchmarking approach
We benchmarked the performance of image streaming to quantify its benefits, using an internal benchmarking suite to compare historical image streaming data versus the latest version. The benchmark is based on the popular Triton Inference Server image, which we use to measure image loading performance.
Figure 1 Startup latency in ms: Blue GA image streaming, Green: Performance improved version
In general, we can see a up to ~30% improvement on the image streaming performance with the newly added enhancements. For large container images such as AI/ML models (e.g., vLLM-based containers, which often start at 8 GB and that can easily be as big as 100GB when you include model weights), this enhancement makes a big difference. By improving image pulling time, you get overall quicker container startup compared to non-streamed containers. These performance enhancements also play a big role in scaling out containers using horizontal pod autoscaler (HPA).
Get started with faster image streaming today
These image streaming performance improvements are automatically available to you when using GKE versions 1.32.1-gke.1729000 or newer. If you are already using container image streaming, there are no configuration changes you need to make to benefit from these enhancements. If you are not yet using image streaming, simply enable it on any new or existing GKE cluster to instantly get these benefits.
Improved image streaming performance marks another milestone on our journey to provide you with the fastest and most efficient container management platform. We will roll out further improvements to image streaming focused on image availability, usability, reliability, and integration with other GKE capabilities.
We encourage you to leverage these new enhancements to accelerate your application deployments on GKE. Check out the image streaming documentation to get started today and tap into a world of applications that start and scale faster!