Helicone vs Promptfoo

In short

• Both: Choice of models, Self-hosted, API, Traces and evaluates
• Only Helicone states: Official SDKs
• Only Promptfoo states: Command line

Helicone

Helicone routes, monitors, debugs, and analyzes AI applications through an open-source gateway and observability platform.

Plans

Hobby

Free

• 10,000 free requests
• 1 GB storage
• 1 seat, 1 organization

Pro

• $79 per month

• Everything in Hobby
• Unlimited seats
• Alerts & reports
• HQL (Query Language)

Team

• $799 per month

• Everything in Pro
• 5 organizations
• SOC-2 & HIPAA compliance
• Dedicated Slack channel

Enterprise

Price on request

• Everything in Team
• Custom MSA
• SAML SSO
• On-prem deployment
• Bulk cloud discounts

Prices checked 2026-09-25 on the maker’s page.

Capabilities

• Choice of models — “Request Routing : Route by model, cost, or custom rules” source
• Self-hosted — “Now you can deploy our powerful observability platform directly within your own infrastructure with a single Docker command.” source
• API — “With this setup, any calls to the OpenAI Responses API will be automatically logged and monitored by Helicone.” source
• Official SDKs — “We’re thrilled to announce that we now have a Go SDK for Helicone’s Helpers Package .” source
• Traces and evaluates — “Built-in Observability : Full integration with Helicone’s analytics” source

Latest updates

• 2025-11-26 Claude Sonnet 4 and Sonnet 4.5 now support 1M context window

  Sonnet 4 and Sonnet 4.5 models on the AI Gateway now support a 1M token context window by default across Anthropic API, AWS Bedrock, and Google Vertex AI.

• 2025-08-13 Control Reasoning Effort in Playground and better feedback on thinking models

  Added the reasoning effort parameter, a minimal option, existing reasoning levels, and visual reasoning display when available.

• 2025-08-07 OpenAI GPT-5 Models Pricing and Playground Support

  Added pricing for GPT-5, GPT-5-Mini, GPT-5-Nano, and GPT-5-Chat-Latest models across multiple providers.

Promptfoo

Promptfoo evaluates and red-teams LLM apps, prompts, models, agents, and RAG systems through a CLI, library, and web interface.

Plans

Community

Free Forever

• All LLM evaluation features
• All model providers and integrations
• Red teaming (10k probes/month)
• Custom integration with your own app
• Run locally or self-host on your own infrastructure
• Vulnerability scanning
• Community support

Enterprise

Custom

• Custom red teaming limits
• Team sharing & collaboration
• Continuous monitoring
• Centralized security/compliance dashboard
• SSO and granular permission profiles
• Promptfoo API access
• Managed cloud deployment
• Priority support & SLA guarantees

On-Premise

Custom

• All Enterprise features
• Deployment on your own infrastructure
• Complete data isolation
• Dedicated runner
• Assigned deployment engineer

Prices checked 2026-09-24 on the maker’s page.

Capabilities

• Command line — “promptfoo is an open-source CLI and library for evaluating and red-teaming LLM apps.” source
• Choice of models — “Use OpenAI, Anthropic, Azure, Google, HuggingFace, open-source models like Llama, or integrate custom API providers for any LLM API” source
• Self-hosted — “Run locally or self-host on your own infrastructure” source
• API — “Promptfoo API access” source
• Traces and evaluates — “Test and evaluate your prompts, models, and RAG pipelines” source

Security

• SOC 2 Type II — “Promptfoo SOC2 Type II Report” source
• SOC 2 — “Promptfoo SOC2 Type II Report” source
• ISO 27001 — “Promptfoo ISO 27001 Certificate” source

Latest updates

• 2026-03-03 Open-Sourcing ModelAudit: Security Scanner for ML Model Files

  ModelAudit was released as an MIT-licensed open-source static scanner for ML model files.

• 2025-12-16 Building a Security Scanner for LLM Apps

  Promptfoo is adding code scanning for LLM-related vulnerabilities, first as a GitHub Action that reviews pull requests.

• 2025-12-11 GPT-5.2 Initial Trust and Safety Assessment (GPT-5.2)

  Promptfoo opened a PR for GPT-5.2 support.