The Pulse
AWS Adds Usage-Based Billing for SageMaker Streaming Models
Model providers can report usage during SageMaker AI bidirectional streaming sessions for AWS Marketplace software charges. Deepgram is the launch partner, while endpoint infrastructure charges remain separate.
AI.info Team · Written with AI · fact-checked ·
Model providers can now report usage during Amazon SageMaker AI bidirectional streaming sessions for AWS Marketplace software charges. In a Nov. 11 post, Amazon Web Services said the metering lets providers charge for reported consumption during a live stream rather than relying solely on hourly software pricing.
Deepgram is the launch partner, bringing usage-based billing for speech-to-text and text-to-speech models to AWS Marketplace. Charges for the SageMaker AI endpoint infrastructure remain separate and continue while the endpoint is provisioned.
Billing for a live session
For conventional real-time request-response inference, model providers can report a unit count in a response header for the Marketplace software charge. In a bidirectional session, usage accumulates while the stream runs, and the final total is not known until the session ends.
Hourly software pricing can charge a customer while a model is available, including during quiet periods. Under the new approach, the provider defines what counts as metered usage, and the Marketplace offer maps the unit to a price.
For example, a product charging per second of audio could report 12 minutes of metered audio as 720 units. An open connection lasting 12 minutes does not by itself establish the quantity of billable audio; the provider's counting rules determine that quantity.
Marketplace listings show one price per unit, labeled per request, with up to three decimal places. Providers can also extend private offers with negotiated per-unit rates or fixed-price contracts.
A separate metering connection
The system uses a companion WebSocket connection alongside the existing inference stream. Once a model server opts in through a response header, SageMaker AI opens the additional connection for metering data. The two connections share a request identifier that the server uses to match metering records to the inference session.
Each text frame carries exactly one metering record. The metadata stream has a quota of one message per second per connection and a maximum message size of 512 bytes.
AWS recommends sending records incrementally as usage accrues to reduce the amount left unreported if a session ends unexpectedly. Transport delivery alone does not establish that usage has been billed; the company says providers should check reported units against Marketplace reports and buyer charges.
Failures and deployment
If the metadata connection fails to start or drops during a session, inference continues by default without that metering stream. Providers can instead configure the connection as required, which stops inference if the metadata stream becomes unavailable. Neither setting guarantees billing for malformed or dropped records.
For a Deepgram deployment, the model server calculates metering automatically, and SageMaker AI and AWS Marketplace process it without client-side metering configuration. Customers subscribe to a model through Marketplace, deploy its package to a SageMaker AI real-time endpoint and connect using the bidirectional streaming API. The example product named in the post is Deepgram Nova-3 Monolingual Speech-to-Text Streaming.
Model providers choosing per-inference pricing set one per-unit price for real-time inference across instance types. A released listing cannot switch between hourly and per-inference pricing. AWS recommends testing with a limited listing and an allowlisted buyer, including checks of interrupted sessions, metadata-stream loss and malformed messages.