AI Tools
Text Generation Inference
Text Generation Inference: Preise, Fähigkeiten, Updates und Alternativen, von den Seiten des Anbieters gelesen.

Auf Englisch
Key facts
- Price
- Not stated on the maker’s pages
- Free plan
- Not stated on the maker’s pages
- Platforms
- Linux · CLI · API
- API
- Yes
- Open source
- Yes · Apache-2.0 Read on github.com
- Latest version
- v3.3.7
- Last update
- 19 Dec 2025 Read on github.com
Hugging Face’s Text Generation Inference (TGI) runs language models on your own infrastructure. It serves generated text through an API and includes features for handling multiple requests and streaming output.
It is for developers and teams deploying open-source language models. The project is in maintenance mode; Hugging Face recommends considering alternatives such as vLLM, SGLang, llama.cpp, or MLX. Compute and hosting costs are separate and depend on where it runs.
- Category
- Developer Tools
- Read from the maker’s pages
- 2026-11-18
Company
Features
- Serves language models through REST and gRPC interfaces
- Batches incoming requests continuously
- Streams generated tokens using Server-Sent Events
- Uses tensor parallelism to run inference across multiple GPUs
- Supports quantization methods including bitsandbytes and GPTQ
- Loads model weights using Safetensors
- Provides OpenTelemetry tracing and Prometheus metrics
- Open-source under the Apache License 2.0
Use cases
- Deploy an open-source language model behind an application API
- Serve concurrent text-generation requests with continuous batching
- Stream model output to a chat interface as tokens are generated
- Run inference across multiple GPUs for a large language model
- Monitor a model-serving deployment with tracing and metrics
Pricing
Free
Free
- Unlimited public/private repositories
- Dependabot security and version updates
- 2,000 CI/CD minutes/month
- 500MB of Packages storage
- Issues & Projects
- Community support
Team
$4/user/mo
- Access to GitHub Codespaces
- Repository rules
- Multiple reviewers in pull requests
- Draft pull requests
- Code owners
- Required reviewers
Enterprise
$21/user/mo
- GitHub Copilot access
- Agent governance
- Data residency
- Enterprise Managed Users
- User provisioning through SCIM
- Enterprise Account to centrally manage multiple organizations
Capabilities
- Command line — “To see all options to serve your models (in the code or in the cli):” source
- Choice of models — “text-generation-launcher --model-id mistralai/Mistral-7B-Instruct-v0.2” source
- Self-hosted — “You can also opt to install text-generation-inference locally.” source
- API — “You can consult the OpenAPI documentation of the text-generation-inference REST API using the /docs route.” source
- Official SDKs — “clients/ python” source
- Runs models for you — “TGI is a toolkit for deploying and serving Large Language Models (LLMs).” source
- Traces and evaluates — “Production ready (distributed tracing with Open Telemetry, Prometheus metrics)” source
Latest updates
- Version 3.3.7
Added support for max_image_fetch_size and announced maintenance mode.
- Version 3.3.6
- Version 3.3.5
Added Gemma 3 sliding-window support and XPU LoRA support.
- Version 3.3.4
Added initial Gaudi support for Gemma 3 text and VLM models.
- Version 3.3.3
Enabled the Qwen3 MoE model on Gaudi.
Where it works
Linux · CLI · API
Alternatives to Text Generation Inference
- vLLM
vLLM runs and serves large language models on supported hardware, with APIs for applications to send inference requests.
- LoRAX
LoRAX is an open-source inference server for serving multiple fine-tuned LoRA models through shared GPU infrastructure.
- llama.cpp
Run language and vision-language models on CPU or GPU with a C/C++ inference runtime and optional API server.
- Hugging Face
Hugging Face is a platform for finding, sharing, and building with machine-learning models, datasets, and apps.
- Fireworks AI
Fireworks AI hosts, routes, and trains open and custom models for applications, teams, and research.
- Ludwig
Open-source framework for training, fine-tuning, evaluating, and serving AI models with YAML configurations.
Questions about Text Generation Inference
Does Text Generation Inference have an API?
Yes. The maker’s pages state that Text Generation Inference has an API. Read on the maker’s pages on 18 Nov 2026.
Which platforms does Text Generation Inference run on?
Text Generation Inference runs on Linux, CLI and API, as its maker’s pages state. Read on the maker’s pages on 18 Nov 2026.
Is Text Generation Inference open source?
Yes. The code of Text Generation Inference is published at github.com/huggingface/text-generation-inference under the Apache-2.0 licence. Read on the maker’s pages on 18 Nov 2026.
Is Text Generation Inference still available?
Yes, as of 18 Nov 2026: its maker’s pages were read that day and carried no notice that Text Generation Inference has ended.
What changed recently in Text Generation Inference?
The latest entry in its maker’s changelog: 19 Dec 2025, “Version 3.3.7”. Before it: 17 Sept 2025, “Version 3.3.6”. Changelog checked on 16 Nov 2026.
Sources
- https://huggingface.co/pricing — Official Hugging Face pricing page; lists hosted compute pricing, not TGI software tiers.
- https://huggingface.co/docs/text-generation-inference/main/index — Product description, capabilities, and maintenance-mode notice.
- https://github.com/huggingface/text-generation-inference — Repository details, deployment instructions, and archived/read-only status.
- https://github.com/huggingface/text-generation-inference/releases — Release page identifies v3.3.7 as the latest release and describes its changes.
- https://github.com/huggingface/text-generation-inference/blob/main/LICENSE — Apache License 2.0.
Not confirmed
- Prices
- the official pricing page lists Hugging Face hosted compute, not TGI software tiers; no TGI-specific price was found.
- Free plan
- no TGI subscription or hosted-service tier structure was found.