Skip to content
AI.info

tools

Truss

Truss packages, configures, and deploys machine-learning models as production APIs through Baseten.

Truss

Truss is an open-source CLI and model-serving framework for data scientists and machine-learning engineers. It defines model dependencies, runtime settings, hardware, weights, and optional Python serving logic, then deploys models to Baseten.

It supports configuration-only models, custom Python models, and custom Docker servers, including models built with frameworks such as vLLM, SGLang, TensorRT-LLM, PyTorch, and TensorFlow. Truss itself is free, but Baseten hosting, GPU compute, model APIs, and training are billed separately.

Features

  • Packages model code, weights, dependencies, and runtime configuration
  • Deploys configuration-only models from config.yaml
  • Supports custom Python Model classes with load and predict methods
  • Supports custom Docker servers and existing model containers
  • Provides CLI commands for local development, deployment, and promotion
  • Supports live reload and development deployments
  • Works with vLLM, SGLang, TensorRT-LLM, PyTorch, and TensorFlow
  • Open-source under the MIT License

Use cases

  • Package a trained model as a shareable serving artifact
  • Deploy an open-source model as a production API
  • Iterate on model code with live reload during development
  • Bring an existing vLLM or SGLang container to deployment
  • Configure GPU hardware, dependencies, and model weights
  • Promote a tested model deployment to production

Pros

    Cons

      Latest updates

      • v0.18.38 (v0.18.38)

        Added persistent_home to training CacheConfig, optional project accelerator support, and named non-spot capacity standard.

      • joey-1 decision model available on Baseten

        joey-1 is now available through Baseten Model APIs, and its weights are open source on Hugging Face.

      • v0.18.37 (v0.18.37)

        Routed base image builds through Build Platform, accepted list-form config secrets, and changed rebuild_free image and install handling.

      • elephant-1 and mammoth-1 pricing

        elephant-1 and mammoth-1 are moving to paid usage on Baseten Model APIs starting October 27, 2026, at 11:59 p.m. PT.

      • Per-replica metrics

        Compare CPU, GPU, and memory usage across individual replicas in every workspace.

      Capabilities

      • Command line — “Use the Baseten CLI to deploy and manage the package on Baseten with autoscaling and observability.” source
      • API — “The serving configuration is separate from how you deploy it. Deploy a model shows the same model deployed from Explore, from a Truss project with the Baseten CLI, or through the Management API.” source
      • Builds agents and workflows — “Use Chains when a workflow contains multiple models or steps that need different hardware.” source

      Get it

      Security

      • SOC 2 Type II — “Baseten Labs, Inc. SOC 2 Type 2 Report 5.31.26.pdf” source
      • SOC 2 — “Baseten Labs, Inc. SOC 2 Type 2 Report 5.31.26.pdf” source
      • ISO 27001 — “Baseten Labs, Inc. ISO 27001 Certificate.pdf” source
      • GDPR — “Baseten Data Processing Agreement.pdf” source
      • HIPAA — “Baseten Labs, Inc. HIPAA Report 5.31.26.pdf” source

      Pricing

      Starting price
      Free
      Prices checked
      2026-10-24

      Basic

      Free

      • Deploy custom, fine-tuned, and open-source models
      • Dedicated deployments
      • Model APIs
      • Training
      • Fast cold starts
      • SOC 2 Type II and HIPAA compliant

      Enterprise

      Price on request

      • Full control in your cloud and ours
      • Custom SLAs
      • Self-host deployments
      • On-demand flex compute
      • Use existing cloud commitments
      • Full control over data residency
      Official website