The Pulse
Agent Substrate targets denser Kubernetes workloads
NVIDIA's open-source Agent Substrate runtime can suspend idle AI agents and restore them on shared Kubernetes workers. GPU support is also under development, and the runtime links with the Dynamo inference framework and KAI Scheduler.
AI.info Team · Written with AI · fact-checked ·
A pool for stop-start workloads
Agent Substrate has been donated as an open-source runtime for sandboxed agents to the Cloud Native Computing Foundation’s Sandbox. NVIDIA describes the security-hardened, zero-trust system as designed to run millions of agents and tools at 10 times the density of standard containers.
The runtime is aimed at a pattern in which agents take an action, such as running a command or calling a model, and then wait before their next step. NVIDIA says keeping a separate Kubernetes Pod warm for every session wastes resources during those idle periods, while creating Pods on demand incurs a multi-second startup and loses the agent’s state.
Agent Substrate uses Kubernetes to maintain a pool of ready worker Pods, then maps a larger set of agent environments, which the post calls actors, onto that pool. When an actor pauses, its memory and filesystem are checkpointed from a gVisor or microVM sandbox so the worker can be reused; on its next request, the actor can be restored on an available worker. The workers remain ordinary Kubernetes Pods, subject to cluster scheduling, quotas and placement policies.
Three workloads in focus
Coding assistants, reinforcement-learning post-training and research sandboxes are named as target workloads. Coding agents may alternate between model calls and repository work, including editing files and running builds or tests, while users expect their environment to persist between turns.
In reinforcement-learning post-training, GPU trainers and generation servers run alongside CPU-heavy environments such as code sandboxes, compilers, browsers and simulators. Sandbox demand can arrive in bursts as training rollouts dispatch actions; NVIDIA says a run may grow from a few hundred sandboxes in early experiments to many hundreds of thousands in production. For research, the post describes short-lived GPU sandboxes used to compile CUDA code, benchmark kernels or run numerical calculations.
Some capabilities are still being built
The post describes request parking in the Agent Substrate router: when workers are occupied, a request can wait for one to become free, subject to a bounded wait and a limit on queued requests. NVIDIA says the feature is intended for bursts in coding-assistant use or reinforcement-learning rollouts.
GPU access is under development through Kubernetes Dynamic Resource Allocation, or DRA. The proposed device-pluggability model lets each sandbox container request the devices it needs, and NVIDIA says its GPUs will be the first supported device. Another contribution adds per-container CPU and memory limits for microVM sandboxes; a container that exceeds its memory limit can be stopped without stopping the rest of the sandbox.
A vendor-neutral design is also proposed for suspending and resuming GPU sandboxes. Because a saved GPU state may depend on the device model and sometimes its driver version, the design records device properties with each snapshot and restores the sandbox on a matching device, or uses a clean cold start if none is available.
Connecting agent and model execution
Agent Substrate manages agent environments, while NVIDIA Dynamo coordinates inference across GPU workers. Dynamo’s router can use information about key-value cache locations, GPU workload and agent-session signals to route related model calls to workers that already hold reusable context.
Dynamo can also provide Agent Substrate with estimates such as time to first token, expected completion time, request progress, cache readiness and GPU-pool saturation. The runtime can use those signals to decide whether an actor should stay active, pause during a longer model wait or resume as a response nears completion.
KAI Scheduler places Substrate’s worker Pods alongside GPU trainers and inference servers. NVIDIA describes Agent Substrate as an independent CNCF Sandbox project that integrates with components of its DSX AI Factory Platform, rather than a named DSX OS component. The APIs are still changing, GPU support is being rebuilt on DRA, and NVIDIA says parts of the Dynamo integration remain experimental.