The Pulse
Amazon says autonomous AI agents wrote its AWS Emulator
AWS Emulator lets developers test cloud applications on their own machines, with no additional service fee. Amazon says autonomous agents wrote all its code without human review, using tests checked against the AWS cloud.
AI.info Team · Written with AI · fact-checked ·
A new tool for testing Amazon Web Services applications locally was written entirely by autonomous AI agents, according to an Amazon post by senior principal engineer Matthew Goldberg. AWS released the emulator on November 11, making it available for download.
Goldberg said his team had neither written nor reviewed any of the code. The emulator has no additional service fee, although customers pay for AWS resources they deploy in the cloud.
Its source code and fidelity tests are available on GitHub. The code is published as a public dedication under the Creative Commons CC0 license.
Local testing and limits
Developers can install the emulator from RPM or Debian packages on Linux, or run it in Docker on macOS and Windows. The AWS command-line interface, software development kits and coding agents communicate with it through a local HTTPS endpoint.
When the emulator starts, it generates credentials, a configuration file and a certificate. The product also has an embedded mode for the AWS SDK for Rust, allowing an instance to run inside a test process rather than as a separate network server.
AWS cautions that the emulator does not evaluate identity and access management policies and resource-based policies in the same way as its cloud, so a request denied by AWS might succeed locally. The company advises against using it to test whether an application is secure and recommends testing security settings and the whole application on AWS before production.
Each emulator acts as a single account, with no cross-account or cross-Region behavior. An operation that has not yet been implemented returns an HTTP 501 error.
From experiment to product
Goldberg traced the project to an experiment earlier this year using Kiro, a coding agent. The prompt asked Kiro to write tests based on the AWS command-line interface, iterate until they passed against the real cloud, and then build an in-memory implementation that passed the same tests.
Goldberg said Kiro returned a passable initial version of Amazon Simple Queue Service, or SQS, after 15 minutes. After he refined the prompt, the agent built a broader implementation over a weekend; it had gaps, but his team began using it to test its software the following Monday.
Turning the experiment into a product took longer and required engineering work on guardrails, Goldberg wrote. The current process divides the work among three groups of agents: test writers, issue finders and fixers.
Tests, fixes and release gates
The first group creates scenarios from public AWS API Models and checks them against the actual cloud. An SQS test, for example, can create a queue, send and receive a message, acknowledge it and delete the queue. Amazon says the suite contains tens of thousands of fidelity tests across AWS and continues to grow.
A second group replays the tests against the emulator and files issues when tests fail. Each issue then goes to an agent in the third group, which investigates and edits the code. Changes must clear a gate that checks compilation and includes agent reviews of coding standards, security and whether the change reduced fidelity.
Failed changes return for rework; changes that pass are merged into the main codebase. Goldberg said the system had fixed thousands of issues a day while hundreds of agents worked concurrently.
Merged changes pass through a release pipeline using AWS developer tools, with CodeCommit as the agents’ primary repository. Builds are retested before being merged to a release branch, packaged as a downloadable binary and published to GitHub. The public repository contains filed issues, including those fixed and those still in the backlog.
AWS says each release publishes the number of fidelity tests passed for each service, alongside the total tested. The company notes that a score can fall when tests are added, even if the emulator improves.