Code Execution and Sandboxed Compute

On 22 October 2021 three versions of the npm package ua-parser-js — 0.7.29, 0.8.0 and 1.0.0 — were published with malicious code inside them. GitHub's advisory went out the same day, titled Embedded malware in ua-parser-js. It did not tell users to uninstall the package and move on. It told them: “Any computer that has this package installed or running should be considered fully compromised. All secrets and keys stored on that computer should be rotated immediately from a different computer.” NVD issued CVE-2021-4229 for the same event on 24 May 2022. Base score 8.8 (AV:N/AC:L/PR:N/UI:R/S:U/C:H/I:H/A:H), for a backdoor alongside a crypto-mining component. The fixed versions are 0.7.30, 0.8.1 and 1.0.1.

That advisory is the blast radius of a single install command, written out by the people who had to answer for it. Not remove the dependency. Rotate everything the machine could see, and do the rotating from a machine that never touched it. An agent that may install what it likes into an environment holding credentials is one publish away from being that machine.

The dangerous code need not be in the library's source at all. On 29 March 2024 a PostgreSQL developer, Andres Freund, wrote to the oss-security mailing list under the subject "backdoor in upstream xz/liblzma leading to ssh server compromise". His finding: “The upstream xz repository and the xz tarballs have been backdoored.” NVD gave CVE-2024-3094 the maximum base score, 10.0 (AV:N/AC:L/PR:N/UI:N/S:C/C:H/I:H/A:H). The liblzma build extracted a prebuilt object file from a disguised test file and used it to modify functions in the shipped library. Part of the backdoor existed only in the distributed tarballs, not in the repository's tracked sources. A reader auditing the repo would have found nothing. The build step was the payload.

The third piece is supplied by the agent itself. Models invent package names, and a 2025 study measured how often. Its abstract describes the experiment: “Using 16 popular LLMs for code generation and two unique prompt datasets, we generate 576,000 code samples in two programming languages that we analyze for package hallucinations.” At least 5.2% of the packages the commercial models recommended did not exist. For the open-source models it was 21.7%. The names amount to 205,474 unique hallucinated packages. Each one is an unclaimed slot on a public registry, free for anyone to register, under a name some model is already recommending.

Put the three together and the notebook task that "required a small parser" is a complete supply chain. A name the model invented. A registry that will serve whoever claims it. An install step that runs before any test does.

Code execution lets an agent compute precisely, transform data, run tests, and automate work. It also creates a powerful path to files, credentials, networks, subprocesses, and persistence.

A sandbox should define filesystem boundaries, network policy, process limits, dependency sources, execution time, and artifact export. Those six are the contract. Anything left unstated is granted by default.

The model's generated code remains untrusted until the environment contains its effects. Untrusted is not a judgement about the model. It is the only assumption compatible with the measured rate at which generated programs carry weaknesses, and with install steps that run arbitrary code before a single test does.

Generated code escapes through whichever layer nobody specified — the workspace, the runtime image, the capability policy, or the resource limits. Resource limits and evidence should not share an owner or a test.

Five layers, and each one has a default that applies when you decline to choose. The workspace is an isolated filesystem with explicit inputs and export paths. The runtime image is a pinned language, system library, and dependency environment. The capability policy sets rules for network, subprocesses, devices, secrets, and persistence. Resource limits cover CPU, memory, storage, time, and process quotas. Evidence is logs, tests, diffs, artifacts, and environment metadata.

The capability layer is the one people believe they already have. Container runtimes ship the opposite default, and the federal guidance has said so since 2017. NIST Special Publication 800-190, the Application Container Security Guide, lists unbounded network access from containers among its container risks. The default, in its words: “By default in most container runtimes, individual containers are able to access each other and the host OS over the network.” The same guide warns that containers do not offer as clear and concrete a security boundary as a VM, because they share a kernel.

That is a federal standard describing the shipped default, not a hypothetical oversight. A team that runs a coding agent "in a container" and never wrote a network policy has not restricted its outbound reach. It has accepted the runtime's. The runtime's is open — to the other containers and to the host.

An ephemeral sandbox, a persistent workspace, and host execution trade convenience against blast radius, in that order. The question to ask of each is the same. After a run, what could the code have touched that the task never mentioned?

An ephemeral sandbox answers easily, because whatever it touched is discarded with the container: strong isolation, simple cleanup, harder long-running state. A persistent workspace answers only if someone keeps maintaining the boundary. Files survive across steps or sessions, which supports projects but needs provenance and quotas, and lets attacks accumulate. Host execution runs generated code near production resources, and largely cannot answer at all: low setup cost, large blast radius, a poor default.

The usual argument for the last two is that real isolation is too expensive to put around every run. That price is published. Amazon Web Services reported Firecracker's numbers in 2020: memory overhead of less than 5MB per container, boot to application code in less than 125ms, and creation of up to 150 MicroVMs per second per host. Its jailer wraps the VMM in a chroot with pid and network namespaces, drops privileges, and applies a restrictive syscall filter: “The seccomp-bpf profile whitelists 24 syscalls, each with additional argument filtering, and 30 ioctls (of which 22 are required by KVM ioctl-based API).”

Under 5MB, under 125ms, 150 a second, and 24 permitted syscalls. Whatever argues for running generated code on the host, it is no longer the cost of the alternative. The choice is right when generated code cannot access secrets or networks outside the task policy, whatever the convenience it costs to get there.

Threat-model the workspace a coding agent already uses. Its filesystem, its network, its package sources, its export path. For each of those four, write down what the agent can reach today and what the task actually requires, then close the gap between the two. If the exercise ends with no gap anywhere, suspect the threat model rather than the workspace, and try it again with a hostile reader. Record what you found. The next person to look will otherwise start from the same blank page.

Five steps make the result auditable. List assets: identify source code, secrets, customer data, networks, and build credentials. Define isolation: choose filesystem, process, network, and identity boundaries. Pin dependencies: use approved registries, lockfiles, hashes, and immutable images. Collect evidence: capture commands, outputs, diffs, tests, and exported artifacts. Gate promotion: require review or deterministic checks before code leaves the sandbox.

The third step is the one that gets waved through, and it is the one with a number attached. A 2019 study called Small World with High Risks went through the npm ecosystem: 5,386,239 package versions, 199,327 maintainers, 609 publicly reported security issues. Its finding: “Installing an average npm package introduces an implicit trust on 79 third-party packages and 39 maintainers, creating a surprisingly large attack surface.” By 2018, 391 maintainers each affected more than 10,000 packages, and 20 colluding maintainers could reach more than half the ecosystem.

So one `install` in an agent workspace is not a decision about one library. It is a trust decision about 79 packages and 39 people, taken by a process that will not be asked to justify it. Lockfiles, hashes, an approved registry and an immutable image are how that decision gets made once, by someone, in advance — rather than freshly at every run, by the model.

A coding agent is safer when it produces a reviewable patch and test evidence than when it receives direct write access to production.

A reviewer's job starts where the suite stops. Passing tests does not prove that generated code is safe, only that it does what the tests thought to ask of it. Roughly 40% of the 1,689 programs in the Copilot study were vulnerable while doing the job asked of them. The install step that shipped the ua-parser-js backdoor ran before any test in the project could have looked at it. So read the patch for what it reaches — files, credentials, outbound calls, new dependencies — and not only for whether the run came back green.

The sandbox is what makes that review affordable, because anything the code did while it ran was already confined to what the task allowed. Without it, the review has to be exhaustive, since a miss is unbounded. With it, a miss costs a discarded container.