Arize Phoenix vs Promptfoo

In short

• Both: Command line, Self-hosted, API, Traces and evaluates
• Only Promptfoo states: Choice of models

Arize Phoenix

Open-source platform for tracing, evaluating, and improving AI agents and LLM applications.

Plans

AX Free

Free

• 10 issues per month
• 25k spans per month
• 1 GB per month
• 15 days retention
• SaaS deployment
• Unlimited users and evals

AX Pro

• $50 per month

• 25 issues per month
• 50k spans per month
• 10 GB per month
• 30 days retention
• SaaS deployment
• Unlimited users and evals

AX Enterprise

Price on request

• Unlimited issues
• Custom span volume
• Custom ingestion volume
• Custom retention
• SaaS or Self-Hosted deployment
• Unlimited users and evals

Prices checked 2026-09-25 on the maker’s page.

Capabilities

• Command line — “Get Started Locally $ uvx arize-phoenix serve” source
• Self-hosted — “Self-host Phoenix and keep sensitive data on your own infrastructure.” source
• API — “Docs Quickstarts, API reference, and integration guides” source
• Traces and evaluates — “Phoenix shows every step your agent takes so you know what went wrong.” source

Security

• SOC 2 Type II — “Arize AI Announces SOC 2 Type II Certification” source
• SOC 2 — “Arize AI Announces SOC 2 Type II Certification” source
• ISO 27001 — “ISO/IEC 27001 Certified” source
• GDPR — “GDPR Compliant” source
• HIPAA — “HIPAA Compliant” source
• Data kept in the EU — “Arize AI: Support for EU Data Residency” source

Latest updates

• 2026-09-17 Get Alyx help tailored to your experience

  Alyx adapts how it helps based on your experience level in each part of Arize AX.

• 2026-09-17 Pick up where you left off with Alyx long-term memory

  Alyx remembers context across conversations, including preferences, goals, project and asset learnings, team conventions, and decisions.

• 2026-09-08 Add and annotate entire sessions and traces in annotation queues

  Send sessions and traces to an annotation queue and review them at session or trace granularity.

Promptfoo

Promptfoo evaluates and red-teams LLM apps, prompts, models, agents, and RAG systems through a CLI, library, and web interface.

Plans

Community

Free Forever

• All LLM evaluation features
• All model providers and integrations
• Red teaming (10k probes/month)
• Custom integration with your own app
• Run locally or self-host on your own infrastructure
• Vulnerability scanning
• Community support

Enterprise

Custom

• Custom red teaming limits
• Team sharing & collaboration
• Continuous monitoring
• Centralized security/compliance dashboard
• SSO and granular permission profiles
• Promptfoo API access
• Managed cloud deployment
• Priority support & SLA guarantees

On-Premise

Custom

• All Enterprise features
• Deployment on your own infrastructure
• Complete data isolation
• Dedicated runner
• Assigned deployment engineer

Prices checked 2026-09-24 on the maker’s page.

Capabilities

• Command line — “promptfoo is an open-source CLI and library for evaluating and red-teaming LLM apps.” source
• Choice of models — “Use OpenAI, Anthropic, Azure, Google, HuggingFace, open-source models like Llama, or integrate custom API providers for any LLM API” source
• Self-hosted — “Run locally or self-host on your own infrastructure” source
• API — “Promptfoo API access” source
• Traces and evaluates — “Test and evaluate your prompts, models, and RAG pipelines” source

Security

• SOC 2 Type II — “Promptfoo SOC2 Type II Report” source
• SOC 2 — “Promptfoo SOC2 Type II Report” source
• ISO 27001 — “Promptfoo ISO 27001 Certificate” source

Latest updates

• 2026-03-03 Open-Sourcing ModelAudit: Security Scanner for ML Model Files

  ModelAudit was released as an MIT-licensed open-source static scanner for ML model files.

• 2025-12-16 Building a Security Scanner for LLM Apps

  Promptfoo is adding code scanning for LLM-related vulnerabilities, first as a GitHub Action that reviews pull requests.

• 2025-12-11 GPT-5.2 Initial Trust and Safety Assessment (GPT-5.2)

  Promptfoo opened a PR for GPT-5.2 support.