tools
HHEM (Hughes Hallucination Evaluation Model)
Open-source model that scores whether an LLM response is factually consistent with a provided source.
HHEM evaluates a source passage and an LLM-generated response, producing a score for factual consistency and hallucination detection. It is designed for developers, researchers, and teams testing summarization, question-answering, and retrieval-augmented generation systems.
The Hugging Face version can be run locally with Transformers and is licensed under Apache-2.0. Vectara also offers a more advanced commercial evaluator through its platform; enterprise deployment pricing applies there.
Features
- Scores factual consistency between a premise and a generated response
- Supports text-classification inference with Transformers
- Runs locally on consumer-grade hardware
- Provides HHEM-2.1-Open as the current open model
- Uses an Apache-2.0 open-source license
- Supports evaluation of summarization and question-answering outputs
- Can be integrated into RAG evaluation workflows
Use cases
- Score generated summaries against their source documents
- Check whether RAG answers are grounded in retrieved context
- Compare hallucination rates across language models
- Evaluate question-answering responses for unsupported claims
- Run factual-consistency checks during model testing
Pros
Cons
Latest updates
Capabilities
- Self-hosted — “HHEM-2.1-Open can be run on consumer-grade hardware” source
- API — “HHEM-2.3 is fully integrated into Vectara and is automatically returned with every query API call.” source
- Runs models for you — “HHEM-2.3 is fully integrated into Vectara and is automatically returned with every query API call.” source
Get it
Pricing
- Starting price
- $100000/yr
- Prices checked
- 2026-11-06
Free Trial
Free
- 30 days to try Vectara
- 10,000 credits included
- All features included for 30 days
SaaS
- Starting at $100K/ year*
- 1 SaaS Deployment
- 10,000,000 credits/year included with base platform package purchase
VPC
- Starting at $250K/ year*
- 1 VPC Deployment (Any VPC)
- 100,000,000 credits/year included with base platform package purchase
On-prem
- Starting at $500K/ year*
- 1 On-Premise Deployment
- 100,000,000 credits/year included with base platform package purchase