Docker Lazy Loading Explained: Faster Container Startup with eStargz and SOCI

Why Container Startup Time Matters

Container images for data platforms tend to be heavy. Images for tools such as Apache Airflow or Spark Connect can easily reach two gigabytes, and on a node that has never seen the image, pulling and unpacking it can take minutes. That delay is invisible when a service runs steadily, but it hurts exactly when speed matters most: during traffic spikes, autoscaling events and node replacements. Pods sit in a pending state, new capacity arrives late, and compute is paid for while nothing useful happens.

Grab's data platform team faced this problem and tested lazy loading as a remedy, comparing two technologies, eStargz and Seekable OCI (SOCI). Their results make a useful case study, but to understand why the numbers look the way they do, it helps to first look at how images normally get onto a node.

How Images Normally Reach a Node

What a Container Sees: rootfs and Layers

Every container has a root filesystem, or rootfs: the directory tree it sees as /. It holds the application, its dependencies, system libraries and configuration, all isolated from the host's own filesystem. The rootfs is not a single blob. It is assembled from the image's layers, each of which records the filesystem changes produced by one build step in the Dockerfile.

OverlayFS and Copy-on-Write

In containerd, a component called a snapshotter turns those layers into a usable rootfs. The default one, overlayFS, relies on the Linux kernel's OverlayFS driver, which works with three directories:

• lowerdir: the stack of read-only image layers, never modified
• upperdir: a fresh, empty, writable directory for this specific container
• merged: the unified view of both, which the container uses as its rootfs

Reads are served from the merged view. When the container writes or modifies a file, the change lands in the upperdir through copy-on-write, so the image layers stay untouched and can be shared by many containers. This is efficient once the layers are on disk. The catch is getting them there.

The Cost of Pulling Everything First

A conventional image pull goes through four strictly sequential steps: download every layer tarball, decompress and extract each one onto local disk, let the snapshotter combine them into a rootfs, and only then start the container. Startup time therefore grows with image size, even if the application only reads a small fraction of the files at boot. For multi-gigabyte images, most of the waiting is spent on data the process may not touch for minutes, or ever.

Lazy Loading with Remote Snapshotters

Lazy loading flips that order. A remote snapshotter creates a snapshot that points at the layer data where it already lives, in the container registry, instead of requiring everything locally. The container can start as soon as the filesystem structure is known, and file contents are fetched on demand when the process first reads them.

Where overlayFS uses directories on local disk as its lowerdir, a remote snapshotter provides a virtual lowerdir backed by the registry. This is typically implemented with FUSE (Filesystem in Userspace): the snapshotter exposes the remote layer's contents as if they were a local directory, and that FUSE mount becomes the lowerdir for the regular OverlayFS driver. Copy-on-write and the merged view keep working exactly as before; only the source of the read-only data changes. From the container's point of view nothing is different: it sees an ordinary directory tree and issues ordinary system calls, unaware that some of those reads are being satisfied over the network.

For this to work, the snapshotter must be able to jump directly to a single file inside a compressed layer. Standard gzip layers do not allow that, which is where eStargz and SOCI come in.

eStargz: Making Layers Seekable

eStargz is an extension of the regular OCI tar.gz layer format that stays backward compatible, so runtimes without lazy loading support can still pull and run these images normally.

Inside an eStargz Layer

The format adds a few structural features:

• Per-file compression. Each file, and each chunk of a large file, is compressed separately, which makes random access possible.
• A table of contents. A JSON index named stargz.index.json, stored near the end of the layer, lists every file with its name, size and byte offset inside the blob.
• A footer. The last few bytes of the layer record where the table of contents starts, so it can be located with one small read.
• Chunk digests. Every chunk carries its own digest in the table of contents, so each downloaded piece can be verified independently.
• A prefetch landmark. A special .prefetch.landmark entry marks the end of a group of priority files that the workload needs first.

What Happens at Container Start

When containerd asks the stargz snapshotter to mount a layer, the snapshotter fetches the footer and then the table of contents from the registry. With that metadata in memory, it mounts a FUSE filesystem, and the container can start even though almost none of the file data has been downloaded.

From then on, every read is intercepted by FUSE. The snapshotter checks its local disk cache; on a miss, it sends an HTTP Range request to the registry for just the bytes it needs, returns them to the process and caches them for later reads. Meanwhile a background task prefetches the priority files up to the landmark and can optionally pull the rest of the layer, so the cache warms up while the application is already running.

SOCI: An Index Instead of a New Format

SOCI, open sourced by AWS, borrows several ideas from stargz but takes a different route. Rather than changing the image itself, it builds a separate SOCI index describing the files inside an existing image. The index is stored in the registry as an OCI artifact and linked to the image through OCI reference types.

This design has practical advantages. Images do not need to be rebuilt or converted, their digests stay the same, and existing image signatures remain valid. SOCI is also supported natively by Bottlerocket, AWS's container-focused operating system, which turns out to matter a great deal operationally.

Benchmarks on Cold Nodes

Grab first measured both approaches on clean nodes with no cached images, comparing them to the default overlayFS snapshotter:

NVIDIA Makes the Run:ai Kubernetes GPU Orchestration Platform Fully Open Source

NVIDIA has released Run:ai, the enterprise GPU orchestration platform it acquired, as open source under Apache 2.0. Here are its 4 modular components and features.

What Is a DevOps Platform? 10 Tools Compared

What a DevOps platform does, how it helps development and operations teams work together, and how ten popular platforms compare.

Pull Time vs Startup Time

eStargz posts the most dramatic pull times, under ten seconds in both cases, but application startup then takes four to five times longer than usual, because the process keeps stalling on files that still have to be fetched. SOCI's pull phase is longer, yet application startup stays identical to the overlayFS baseline.

The lesson is that lazy loading does not make downloading free; it moves the cost from before startup to during runtime. SOCI's separate index and its fetching strategy strike a better balance between the two phases, which is why it ends up with the best or equal total time while leaving application behavior unchanged.

Production Results

After rolling SOCI out in production, Grab reported that both Airflow and Spark Connect started roughly 30 to 40 percent faster overall. Looking at P95 startup time:

• Airflow: about 125 s before, about 85 s after, roughly 32% better
• Spark Connect: about 103 s before, about 68 s after, roughly 34% better

These P95 figures cover the entire path from pulling the image to the application being up, across both clean and warm nodes, so they reflect real-world system behavior rather than a best-case cold start. In practice the improvement translates into faster autoscaling reactions, less compute wasted during startup and a better experience for teams running data jobs. The gains held steady over time, which suggests lazy loading is a stable production optimization rather than a fragile trick.

Tuning Parallel Pull Settings

Default settings were not optimal. Following AWS guidance for SOCI's parallel pull mode on EKS, the team raised three snapshotter parameters:

• max_concurrent_downloads_per_image from 5 to 10
• max_concurrent_unpacks_per_image from 3 to 10
• concurrent_download_chunk_size from 8 MB to 16 MB, in line with AWS's recommendation for Amazon ECR

That change alone brought image download time on a clean node from about 60 seconds to about 24, a 60% improvement. Defaults are a starting point, and parameters should be tuned for the specific registry and network setup.

Setting It Up on EKS

Enabling a Snapshotter

On EKS nodes running containerd, the stargz route means installing the containerd-stargz-grpc daemon and registering it with containerd as a proxy plugin. The configuration looks roughly like this:

[proxy_plugins.stargz]
  type = "snapshot"
  address = "/run/containerd-stargz-grpc/containerd-stargz-grpc.sock"

For production, the installation is usually baked into a custom AMI; for experiments, a node bootstrap script in user data is enough. On Bottlerocket, SOCI needs no extra daemon at all; a single user data setting switches the runtime to it:

[settings.container-runtime]
snapshotter = "soci"

Producing Lazy-Loadable Images

eStargz images can be built with Docker Buildx by selecting estargz compression on output:

docker buildx build \
  --output type=registry,oci-mediatypes=true,compression=estargz,force-compression=true \
  -t registry.example.com/app:1.0 .

SOCI requires no rebuild, only an index for existing images. Docker cannot generate SOCI indexes natively yet, so common options are AWS's Lambda-based SOCI Index Builder, which creates indexes when images are pushed to ECR, or adding an index-generation step to the CI/CD pipeline.

Why SOCI Won for Production

Grab's exploration started with eStargz, but SOCI became the production choice mainly for operational reasons. The company already relied on Bottlerocket to improve pod startup and security at scale, and native SOCI support there means service teams do not have to install and maintain the more complex stargz snapshotter across every EKS cluster. That makes the setup easier to operate and better supported by AWS. Add the benchmark finding that SOCI keeps application startup on par with normal images, and avoiding image conversion altogether, and the decision becomes straightforward.

Key Lessons

• On clean nodes, lazy loading made image availability roughly four times faster.
• P95 startup for production workloads improved by about 29 to 34 percent.
• Tuning SOCI's parallel pull parameters cut download time by a further 60 percent.
• Lazy loading shifts download cost into runtime rather than eliminating it, so application latency should be monitored after rollout.
• Platform fit matters as much as raw speed: native OS support and no image conversion made SOCI the lower-maintenance option.

Lazy loading is not a universal win, though. Small images that already pull in a few seconds gain little, and workloads that read most of their files immediately at boot will end up fetching nearly the whole image anyway, just in many small requests instead of a few large ones. Registry proximity and bandwidth also matter: on-demand reads are only as fast as the round trip to the registry, so a registry in the same region as the cluster is close to a requirement. A sensible rollout therefore starts with the largest, most frequently scaled images, measures pull time, application start and steady-state latency separately, and only then expands to more services.

For teams running large images on Kubernetes, especially with bursty autoscaling, lazy loading is a mature, low-risk optimization built on standard components, and one worth benchmarking against your own workloads.