tools
HHEM (Hughes Hallucination Evaluation Model)
Open-source model that scores whether an LLM response is factually consistent with a provided source.
HHEM evaluates a source passage and an LLM-generated response, producing a score for factual consistency and hallucination detection. It is designed for developers, researchers, and teams testing summarization, question-answering, and retrieval-augmented generation systems.
The Hugging Face version can be run locally with Transformers and is licensed under Apache-2.0. Vectara also offers a more advanced commercial evaluator through its platform; enterprise deployment pricing applies there.
Features
- Scores factual consistency between a premise and a generated response
- Supports text-classification inference with Transformers
- Runs locally on consumer-grade hardware
- Provides HHEM-2.1-Open as the current open model
- Uses an Apache-2.0 open-source license
- Supports evaluation of summarization and question-answering outputs
- Can be integrated into RAG evaluation workflows
Use cases
- Score generated summaries against their source documents
- Check whether RAG answers are grounded in retrieved context
- Compare hallucination rates across language models
- Evaluate question-answering responses for unsupported claims
- Run factual-consistency checks during model testing
Pros
Cons
Pricing
- Starting price
- Starting at $100K/ year
- Pricing checked
- 2026-09-19
30 Day Free Trial
30 Day Free Trial
- All features included for 30 days
SaaS
Starting at $100K/ year
- 1 SaaS deployment
VPC
Starting at $250K/ year
- 1 VPC deployment (any VPC)
On-prem
Starting at $500K/ year
- 1 on-premise deployment