Tools
LoRAX vs Text Generation Inference
LoRAX or Text Generation Inference? Their plans monthly and yearly, the capabilities their makers state, security and latest updates, side by side — read from the makers' own pages.
In short
- Both: Choice of models, Self-hosted, API, Runs models for you, Traces and evaluates
- Only LoRAX states: Official SDKs
- Only Text Generation Inference states: Command line
LoRAX
LoRAX is an open-source server for serving thousands of fine-tuned LoRA models on shared GPU infrastructure.
Plans
Not read from the maker’s page yet.
Prices checked 2026-09-25 on the maker’s page.
Capabilities
- Choice of models — “LoRAX supports a number of Large Language Models as the base model including Llama (including CodeLlama ), Mistral (including Zephyr ), and Qwen .” source
- Self-hosted — “LoRAX (LoRA eXchange) is a framework that allows users to serve thousands of fine-tuned models on a single GPU, dramatically reducing the cost of serving without compromising on throughput or latency.” source
- API — “LoRAX supports multi-turn chat conversations combined with dynamic adapter loading through an OpenAI compatible API.” source
- Official SDKs — “For a full tutorial including token streaming and the Python client, see Getting Started - Docker .” source
- Runs models for you — “Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs” source
- Traces and evaluates — “prebuilt Docker images, Helm charts for Kubernetes, Prometheus metrics, and distributed tracing with Open Telemetry.” source
Text Generation Inference
Open-source server for deploying and serving language models with streaming, batching, quantization, and multi-GPU inference.
Plans
Free
Free
- Unlimited public/private repositories
- Dependabot security and version updates
- 2,000 CI/CD minutes/month
- 500MB of Packages storage
- Issues & Projects
- Community support
Team
- $4 USD per user/month
- Access to GitHub Codespaces
- Repository rules
- Multiple reviewers in pull requests
- Draft pull requests
- Code owners
- 3,000 CI/CD minutes/month
Enterprise
- Starting at $21 USD per user/month
- Data residency
- Enterprise Managed Users
- User provisioning through SCIM
- Enterprise account to centrally manage multiple organizations
- Environment protection rules
- Audit Log API
Prices checked 2026-09-25 on the maker’s page.
Capabilities
- Command line — “To see all options to serve your models (in the code or in the cli): text-generation-launcher --help” source
- Choice of models — “text-generation-launcher --model-id mistralai/Mistral-7B-Instruct-v0.2” source
- Self-hosted — “You can also opt to install text-generation-inference locally.” source
- API — “You can consult the OpenAPI documentation of the text-generation-inference REST API using the /docs route.” source
- Runs models for you — “Text Generation Inference (TGI) is a toolkit for deploying and serving Large Language Models (LLMs).” source
- Traces and evaluates — “Production ready (distributed tracing with Open Telemetry, Prometheus metrics)” source
Latest updates
- v3.3.7 (v3.3.7)
Added support for max_image_fetch_size and announced maintenance mode.
- v3.3.5 (v3.3.5)
Added Gemma 3 sliding-window support and XPU LoRA support.
- v3.3.4 (v3.3.4)
Added initial Gaudi support for Gemma 3 text and VLM models.