tools
Opik
Opik is an open-source platform for tracing, evaluating, testing, and monitoring LLM applications and AI agents.
Opik records LLM calls, tool invocations, agent steps, errors, token usage, and costs. Teams use it to debug applications, evaluate outputs with automated metrics, run test suites, manage prompts, and monitor production systems.
It supports hosted cloud plans and self-hosting. The free plans include core observability and evaluation features, while higher tiers add increased span limits, customizable retention, enterprise deployments, SSO, compliance, and dedicated support.
Features
- Trace LLM calls, tool invocations, agent steps, errors, tokens, and costs
- Evaluate applications with LLM-as-a-judge and heuristic metrics
- Create test suites with item-level and suite-level assertions
- Monitor production traces with online evaluation rules and alerts
- Manage prompt versions and compare prompts in the Prompt Playground
- Optimize prompts with six optimization algorithms
- Self-host with Docker or Kubernetes
- Open-source project; license not read
Use cases
- Debug multi-step agents and retrieval-augmented generation pipelines
- Evaluate chatbot answers for relevance, hallucination, and task completion
- Run regression tests before deploying new prompts or models
- Monitor production quality, latency, errors, token usage, and model cost
- Review and annotate traces with subject-matter experts
- Track coding-agent usage and spending across engineering teams
Pros
Cons
Pricing
- Starting price
- Free
- Pricing checked
- 2026-09-19
Open Source
$0
- Full AI observability and agent testing feature set
- Same codebase as hosted versions
- Agent tracing and analysis
- Test Suites and assertions
- Agent Playground
Free Cloud
$0
- Up to 10 team members
- 25k spans per month
- 60-day data retention
- Agent tracing and analysis
- Test Suites and assertions
- Agent Playground
- Ollie coding harness trial
Pro Cloud
$19
- Up to 50 team members
- 100k spans per month
- 60-day data retention
- Customizable monthly span limits
- Customizable data retention periods
Enterprise
Custom
- Unlimited team members
- Custom usage plans
- Flexible deployments
- Service accounts and view-only users
- Single sign-on
- Dedicated support and SLAs
- SOC 2, ISO 27001, ISO 9001, HIPAA, and GDPR compliance