tools
Truss
Truss packages, configures, and deploys machine-learning models as production APIs through Baseten.
In inglese
Truss is an open-source CLI and model-serving framework for data scientists and machine-learning engineers. It defines model dependencies, runtime settings, hardware, weights, and optional Python serving logic, then deploys models to Baseten.
It supports configuration-only models, custom Python models, and custom Docker servers, including models built with frameworks such as vLLM, SGLang, TensorRT-LLM, PyTorch, and TensorFlow. Truss itself is free, but Baseten hosting, GPU compute, model APIs, and training are billed separately.
Features
- Packages model code, weights, dependencies, and runtime configuration
- Deploys configuration-only models from config.yaml
- Supports custom Python Model classes with load and predict methods
- Supports custom Docker servers and existing model containers
- Provides CLI commands for local development, deployment, and promotion
- Supports live reload and development deployments
- Works with vLLM, SGLang, TensorRT-LLM, PyTorch, and TensorFlow
- Open-source under the MIT License
Use cases
- Package a trained model as a shareable serving artifact
- Deploy an open-source model as a production API
- Iterate on model code with live reload during development
- Bring an existing vLLM or SGLang container to deployment
- Configure GPU hardware, dependencies, and model weights
- Promote a tested model deployment to production
Pros
Cons
Latest updates
- Upcoming change to Baseten API key format
Newly created Baseten API keys will add a b10_ prefix in code, logs, and secret scanners.
- Web search with Baseten Hosted Tools
Baseten Hosted Tools now brings server-side web search to Model APIs through Baseten Grounded Inference.
- Baseten CLI 1.0.0 (1.0.0)
The Baseten CLI is now generally available.
- Model API Deprecation (GLM 4.7, Kimi K2.7, Kimi K2.6, Inkling, Inkling Small, DeepSeek v4 Pro)
GLM 4.7, Kimi K2.7, Kimi K2.6, Inkling, Inkling Small, and DeepSeek v4 Pro will be deprecated at 5pm PT September 25th.
- OIDC and AWS AssumeRole for training jobs
Training jobs can use OIDC or AWS AssumeRole to pull private container images and download model weights or training data.
Capabilities
- Command line — “Use the Baseten CLI to deploy and manage the package on Baseten with autoscaling and observability.” source
- Self-hosted — “With Truss, "a model exists" and "I can run the model on any machine" are equivalent statements.” source
- API — “Truss turns your ML model into a backend for web, no Django or Flask needed.” source
- Runs models for you — “Develop and deploy your own models on dedicated infrastructure.” source
- Builds agents and workflows — “Use Chains when a workflow contains multiple models or steps that need different hardware.” source
Get it
Security
- SOC 2 Type II — “Baseten Labs, Inc. SOC 2 Type 2 Report 5.31.26.pdf” source
- SOC 2 — “Baseten Labs, Inc. SOC 2 Type 2 Report 5.31.26.pdf” source
- ISO 27001 — “Baseten Labs, Inc. ISO 27001 Certificate.pdf” source
- GDPR — “Baseten Data Processing Agreement.pdf” source
- HIPAA — “Baseten Labs, Inc. HIPAA Report 5.31.26.pdf” source
Pricing
- Starting price
- Free
- Prices checked
- 2026-09-25
Basic
Free
- Dedicated deployments
- Model APIs
- Training
- Fast cold starts
- SOC 2 Type II and HIPAA compliant
- Email and in-app chat support