tools
RunPod Serverless
RunPod Serverless deploys containerized AI inference endpoints that autoscale GPU workers and bill compute by the second.
RunPod Serverless runs containerized inference workloads behind an API. It scales workers with request demand, can scale to zero when idle, and supports real-time and batch inference.
Developers and AI teams use it to serve image, speech, recommendation, and language models without managing GPU servers. Pricing is usage-based rather than subscription-based; GPU selection, storage, and deployment choices affect total cost.
Features
- Autoscaling GPU endpoints that scale from zero to hundreds of workers
- Per-second billing from worker start to full stop
- FlashBoot cold-start optimization with sub-200ms cold starts on active endpoints
- Batch inference for high-volume jobs
- Up to 8 GPUs behind a single worker
- Deploy with containers or the no-Docker Flash path
- Network storage for model caching and persistent data
- Webhooks, APIs, custom event triggers, logs, metrics, and tracing
Use cases
- Serve image-generation models through an API
- Run speech recognition or transcription workloads
- Deploy language-model inference for applications
- Process large image or document collections with batch inference
- Scale AI agents and recommendation services with request demand
Pros
Cons
Latest updates
- Terraform provider 2.0.0 (2.0.0)
Built on the REST API v2 and adds support for Instant Clusters, global volumes, CPU Pods and endpoints, FlashBoot tiers, and v2 billing.
- Terraform provider 1.x to 2.0.0 migration (2.0.0)
Upgrading from 1.x requires configuration and state changes; 2.0.0 makes no REST v1 calls.
- New Release Runpod SDK 2.0 (2.0)
Flash is now part of the runpod Python package; define queue endpoints, load-balanced APIs, and tasks that run on their own Pod.
- Breaking runpod 2.0 CLI changes (2.0)
runpod 2.0 removes the runpod config, runpod project, and runpod exec commands. Use rp login and rp flash commands.
- Deprecation Standalone Flash package
The runpod-flash package no longer gets new features, and the runpod/flash repository will be archived.
Capabilities
- Command line — “CLI & SDKs. Deploy and manage directly from your terminal.” source
- API — “Serverless GPU endpoints run containerized inference workloads behind an API and scale workers based on demand.” source
- Official SDKs — “CLI & SDKs. Deploy and manage directly from your terminal.” source
- Runs models for you — “Deploy any containerized model as an autoscaling GPU endpoint without managing servers.” source
- Builds agents and workflows — “Build intelligent agent-based systems and workflows.” source
- Traces and evaluates — “Runpod offers a comprehensive monitoring dashboard with real-time logging and distributed tracing for your serverless functions.” source
Get it
Pricing
- Prices checked
- 2026-10-24
B300
- $ 10.65 /hr
- Maximum throughput for big models.
B200
- $ 8.64 /hr
- Maximum throughput for big models.
H200
- $ 5.93 /hr
- Extreme throughput for big models.
RTX 6000 Pro
- $ 3.49 /hr
- PRO
- High throughput for large model inference workloads.
H100
- $ 4.79 /hr
- PRO
- Extreme throughput for big models.
A100
- $ 2.72 /hr
- High throughput GPU, yet still very cost-effective.
L40, L40S, 6000 Ada, MIG 48GB
- $ 1.75 /hr
- PRO
- Extreme inference throughput on LLMs like Llama 3 7B.
A6000, A40
- $ 1.22 /hr
- A cost-effective option for running big models.
5090
- $ 1.58 /hr
- PRO
- Extreme throughput for small-to-medium models.
RTX PRO 4500 Blackwell
- $ 1.15 /hr
- Cost-effective Blackwell inference for 32GB workloads.