companies
Confident AI
Y Combinator-backed team behind DeepEval, the pytest-style open-source framework for unit-testing LLM outputs on hallucination, relevancy and RAG quality.
Jeffrey Ip and Kritin Vongthongsri founded Confident AI in San Francisco in 2024 and went through Y Combinator's Winter 2025 batch. Ip had worked on infrastructure for YouTube Creator Studio at Google and document recommenders at Microsoft; Vongthongsri came from NLP research at Princeton and a background building fintech pipelines. Their main open-source project, DeepEval, treats LLM evaluation like software testing: developers write test cases and attach metrics such as answer relevancy, faithfulness, or G-Eval, then call an assertion that fails the way a normal pytest test would if a response falls below threshold. By 2026 the library covered more than 50 metrics for RAG pipelines, agents, and multi-turn conversations, and was pulling in several million monthly downloads. Confident AI sells a hosted platform built on top of DeepEval for teams that want evaluation dashboards and monitoring without running the open-source library themselves, while keeping the core framework free and open source.
- Founded
- 2024
- Headquarters
- San Francisco, United States
- Sector
- infrastructure
Tools
- DeepEval
Open-source framework for testing and evaluating LLM apps, agents, RAG pipelines, chatbots, and multimodal systems.