# AWS Adds SOCI Lazy Loading to SageMaker Training Jobs

Published 2026-10-28 · AI.info Team · AI.info

> AWS is adding Seekable OCI indexing to Amazon SageMaker AI training and processing jobs, allowing containers to start before their full images download. In AWS benchmarks, a 13.2 GB image started in about six seconds instead of 3 minutes 42 seconds, though the first training epoch took longer as needed files streamed in.

**[Amazon](https://aws.amazon.com/blogs/machine-learning/announcing-soci-indexed-images-for-faster-sagemaker-ai-jobs/) Web Services announced on October 27 that SageMaker AI training and processing jobs can use Seekable OCI (SOCI) indexes to start containers before downloading their full images.** Instead of waiting for every layer to arrive, SageMaker fetches the files needed to start a container and streams the rest as the job uses them. The change requires no edits to a job definition, entry point or data channels.

## Six seconds to start a 13.2 GB image

The delay SOCI targets comes before model code runs: SageMaker jobs ordinarily download, decompress and write the full container image to the instance. Deep-learning images often exceed 10 GB, AWS says, making that wait a recurring part of billable job time. It can take a larger share of short experiments, processing tasks and hyperparameter sweeps made up of many jobs.

In AWS’s benchmark, a 13.2 GB compressed image took a median of 3 minutes 42 seconds to download conventionally and about six seconds with lazy loading. Across eight images ranging from 1.8 GB to 35.4 GB, the company measured SOCI startup times of 3 to 11 seconds, compared with conventional downloads ranging from 48 seconds to 7 minutes 17 seconds. The tests used SageMaker AI jobs on CPU and GPU instances in the US West (Oregon) and US East (N. Virginia) regions.

## The first epoch still pays a cost

Starting the container sooner does not mean every file is already local. AWS found that the first training epoch took 3 to 16 seconds longer with a lazy-loaded image as frameworks fetched files they encountered for the first time; later epochs ran at the same speed as with a full download, within run-to-run variation. Across the training jobs tested, the net result was a finish 1 to 6 minutes sooner, AWS says.

That saving depends on what a job reads before useful work begins. If startup code scans, checksums or preloads most of the image, the first epoch may have to wait for much of the same data a full pull would fetch. AWS’s tests also found a smaller net benefit when workloads loaded large model weights or compiled code at the outset: those files still had to stream before use.

## Pre-indexed images or custom builds

Users can switch to an AWS Deep Learning Container with a “-soci” tag or generate an index for a custom image and store it in Amazon Elastic Container Registry. SageMaker checks for an index when it creates a job; if no index exists, or lazy loading fails, the job uses a standard download instead. The original image digest and signatures remain valid because the index is stored as a separate OCI-compliant artifact.

AWS lists pre-indexed options for PyTorch Training, TensorFlow Training, vLLM and SGLang, with ARM64 variants for some frameworks. Custom indexing requires a Linux host with containerd and the SOCI snapshotter, plus Finch or nerdctl. Support is available in all regions, but images must be in private ECR repositories; images pulled through a private Docker registry using SageMaker’s VPC repository access mode are not eligible.

## Where lazy loading may matter most

The benchmark suggests the strongest case is for large images and jobs that begin work after using only a fraction of their contents. AWS reports comparatively small gains for images below roughly 4 GB. The result is a targeted change to job startup—not a faster training loop—and its practical value will depend on image size, startup behavior and how much of the image a workload needs immediately.

## Sources

1. Original source · Blog post [AWS Machine Learning Blog](https://aws.amazon.com/blogs/machine-learning/announcing-soci-indexed-images-for-faster-sagemaker-ai-jobs/) aws.amazon.com

## Companies in this story

- [Amazon](https://ai.info/companies/amazon)
- [Amazon Web Services](https://ai.info/companies/amazon-web-services)

## Related stories

- [NVIDIA’s 120B Nemotron 3.5 Super VL is available on SageMaker](https://ai.info/news/nvidia-nemotron-35-super-vl-sagemaker-jumpstart)
- [OpenTSLM time-series model becomes available in SageMaker JumpStart](https://ai.info/news/opentslm-time-series-model-sagemaker-jumpstart)
- [AWS Adds Usage-Based Billing for SageMaker Streaming Models](https://ai.info/news/aws-sagemaker-streaming-usage-based-billing)
- [AWS describes checks for silent GPU errors in AI training](https://ai.info/news/aws-sagemaker-hyperpod-silent-gpu-error-checks)
- [AWS details OpenFold3 research preview deployment on SageMaker](https://ai.info/news/aws-openfold3-preview-sagemaker-jumpstart-deployment)

## Explore

[More articles](https://ai.info/news)

---

Canonical: https://ai.info/news/aws-soci-lazy-loading-sagemaker-jobs
