Skip to content
AI.info

tools

Braintrust

Braintrust helps teams trace AI applications, evaluate outputs, find production issues, and improve agents.

Braintrust

Braintrust is an AI observability and evaluation platform for engineering, product, and AI teams. It records agent traces, prompts, responses, tool calls, latency, cost, and quality data.

Teams can run evaluations against datasets, compare prompts and models, score outputs with code, models, or humans, investigate recurring behaviors, and turn production traces into regression datasets. It does not provide a general-purpose AI model; model usage and some data processing or scoring incur additional charges.

Features

  • Inspect prompts, responses, and tool calls in real time
  • Run evaluations against versioned datasets
  • Compare prompts and models side by side
  • Score outputs with LLMs, code, or human reviewers
  • Search production traces and identify recurring behaviors
  • Turn production traces into evaluation datasets
  • Query logs and run evaluations through its MCP server
  • Use SDKs for Python, TypeScript, Go, Ruby, C#, and more

Use cases

  • Trace production AI agents and tool calls
  • Evaluate customer-support responses before release
  • Compare prompts and models against test datasets
  • Find recurring failures across production traces
  • Build regression tests from real failures
  • Monitor latency, cost, and response quality

Pros

    Cons

      Latest updates

      • October 2026

        Added Muse Code session tracing, TypeScript MCP server tracing, and asynchronous SQL queries.

      • October 2026

        Coding agent insights adds usage dashboards and daily Loop analysis; the new thread view groups trace turns into requests, agent work, and responses.

      • October 2026 (v0.22.2)

        Added SQL-query charts with visual editing and Windows DPAPI credential storage shared across terminal and MSIX apps.

      • September 2026 (v0.21.0+)

        Braintrust-hosted deployments map OpenInference traces to structured data; the bt CLI supports local JavaScript span plugins.

      • August 2026

        Monitoring views became dashboards; permissions now control access to organization AI providers; coding-agent tracing runs through the bt CLI.

      Capabilities

      • Choice of models — “Run experiments against real datasets, compare prompts and models side-by-side, and score outputs with LLMs, code, or humans.” source
      • Self-hosted — “Deploy Brainstore data plane on your own infrastructure” source
      • API — “You get a unified API across OpenAI, Anthropic, Google, AWS, and other providers, with automatic caching and observability on every request.” source
      • Official SDKs — “SDKs for Python, TypeScript, Go, Ruby, C#, and more.” source
      • Runs models for you — “Model credits Your monthly model credit covers AI features like Topics, as well as any built-in models you use without your own API key.” source
      • Traces and evaluates — “Inspect prompts, responses, and tool calls in real time” source

      Get it

      Security

      • SOC 2 Type II — “SOC 2 Type II - May 5 - Aug 5, 2025” source
      • SOC 2 — “SOC 2 Type II - May 5 - Aug 5, 2025” source
      • GDPR — “Data Processing Agreement (DPA) - Enterprise” source
      • HIPAA — “One-page white paper on Braintrust and HIPAA compliance.” source

      Pricing

      Starting price
      $0.50/mo
      Prices checked
      2026-10-27

      Starter

      • $0 / month
      • then +$4/GB
      • then $2.50 per 1,000
      • $10 credits / month included
      • 1 GB / month processed data included
      • 10K scores / month included
      • 14-day retention
      • Unlimited users, projects, datasets, playgrounds, and experiments

      Pro

      • $249 / month
      • then +$3/GB
      • then $1.50 per 1,000
      • $0.50 per GB per month
      • $100 credits / month included
      • 5 GB / month processed data included
      • 50K scores / month included
      • 30-day retention
      • Custom charts, environments, priority support, and RBAC

      Enterprise

      Price on request

      • Custom data retention and export
      • RBAC
      • Premium support
      • On-prem or hosted deployment
      Official website