tools
Dataiku LLM Mesh
Dataiku LLM Mesh connects applications to multiple hosted and self-hosted LLMs with centralized routing, governance, monitoring, and cost controls.
Dataiku LLM Mesh provides a governed gateway between applications and LLM providers. Teams can connect hosted APIs or self-hosted models, switch providers without rewriting applications, route requests, and apply access, moderation, and audit policies.
It is used by enterprise AI, data science, IT, and application teams building chatbots, retrieval-augmented generation systems, agents, and other LLM workflows. The product is part of Dataiku and requires a Dataiku deployment; model-provider charges and any applicable Dataiku subscription costs are separate.
Features
- Connect to hosted LLM APIs and locally running Hugging Face models
- Route requests across providers and switch models without rewriting applications
- Apply centralized access controls, usage limits, moderation, and audit policies
- Monitor LLM usage, cost, latency, and round-trip performance
- Cache frequent responses to reduce repeated requests and spend
- Use completion, embedding, streaming, multimodal, and image-generation APIs
- Create persistent multi-turn conversations and query knowledge banks
- Access the LLM Mesh through Python and REST APIs
Use cases
- Route prompts to different models based on cost, performance, or compliance
- Govern access to approved LLM providers across enterprise applications
- Monitor token usage, latency, and provider spend
- Build retrieval-augmented chatbots using internal knowledge banks
- Serve multimodal and image-generation workflows through one API
- Connect Dataiku agents and applications to multiple model providers