Aller au contenu
AI.info

AI Tools

Text Generation Inference

Text Generation Inference : prix, capacités, mises à jour et alternatives, lus sur les pages de l’éditeur.

Le site de Text Generation Inference

En anglais

Key facts

Price
Not stated on the maker’s pages
Free plan
Not stated on the maker’s pages
Platforms
Linux · CLI · API
API
Yes
Open source
Yes · Apache-2.0 Read on github.com
Latest version
v3.3.7
Last update
19 Dec 2025 Read on github.com

Hugging Face’s Text Generation Inference (TGI) runs language models on your own infrastructure. It serves generated text through an API and includes features for handling multiple requests and streaming output.

It is for developers and teams deploying open-source language models. The project is in maintenance mode; Hugging Face recommends considering alternatives such as vLLM, SGLang, llama.cpp, or MLX. Compute and hosting costs are separate and depend on where it runs.

Category
Developer Tools
Read from the maker’s pages
2026-11-18

Company

HuggingFace

Features

  • Serves language models through REST and gRPC interfaces
  • Batches incoming requests continuously
  • Streams generated tokens using Server-Sent Events
  • Uses tensor parallelism to run inference across multiple GPUs
  • Supports quantization methods including bitsandbytes and GPTQ
  • Loads model weights using Safetensors
  • Provides OpenTelemetry tracing and Prometheus metrics
  • Open-source under the Apache License 2.0

Use cases

  • Deploy an open-source language model behind an application API
  • Serve concurrent text-generation requests with continuous batching
  • Stream model output to a chat interface as tokens are generated
  • Run inference across multiple GPUs for a large language model
  • Monitor a model-serving deployment with tracing and metrics

Pricing

Free

Free

  • Unlimited public/private repositories
  • Dependabot security and version updates
  • 2,000 CI/CD minutes/month
  • 500MB of Packages storage
  • Issues & Projects
  • Community support

Team

$4/user/mo

  • Access to GitHub Codespaces
  • Repository rules
  • Multiple reviewers in pull requests
  • Draft pull requests
  • Code owners
  • Required reviewers

Enterprise

$21/user/mo

  • GitHub Copilot access
  • Agent governance
  • Data residency
  • Enterprise Managed Users
  • User provisioning through SCIM
  • Enterprise Account to centrally manage multiple organizations

Capabilities

  • Command line — “To see all options to serve your models (in the code or in the cli):” source
  • Choice of models — “text-generation-launcher --model-id mistralai/Mistral-7B-Instruct-v0.2” source
  • Self-hosted — “You can also opt to install text-generation-inference locally.” source
  • API — “You can consult the OpenAPI documentation of the text-generation-inference REST API using the /docs route.” source
  • Official SDKs — “clients/ python” source
  • Runs models for you — “TGI is a toolkit for deploying and serving Large Language Models (LLMs).” source
  • Traces and evaluates — “Production ready (distributed tracing with Open Telemetry, Prometheus metrics)” source

Latest updates

  • Version 3.3.7

    Added support for max_image_fetch_size and announced maintenance mode.

  • Version 3.3.6
  • Version 3.3.5

    Added Gemma 3 sliding-window support and XPU LoRA support.

  • Version 3.3.4

    Added initial Gaudi support for Gemma 3 text and VLM models.

  • Version 3.3.3

    Enabled the Qwen3 MoE model on Gaudi.

Where it works

Linux · CLI · API

Alternatives to Text Generation Inference

  • vLLM

    vLLM runs and serves large language models on supported hardware, with APIs for applications to send inference requests.

  • LoRAX

    LoRAX is an open-source inference server for serving multiple fine-tuned LoRA models through shared GPU infrastructure.

  • llama.cpp

    Run language and vision-language models on CPU or GPU with a C/C++ inference runtime and optional API server.

  • Hugging Face

    Hugging Face is a platform for finding, sharing, and building with machine-learning models, datasets, and apps.

  • Fireworks AI

    Fireworks AI hosts, routes, and trains open and custom models for applications, teams, and research.

  • Ludwig

    Open-source framework for training, fine-tuning, evaluating, and serving AI models with YAML configurations.

Questions about Text Generation Inference

Does Text Generation Inference have an API?

Yes. The maker’s pages state that Text Generation Inference has an API. Read on the maker’s pages on 18 Nov 2026.

Which platforms does Text Generation Inference run on?

Text Generation Inference runs on Linux, CLI and API, as its maker’s pages state. Read on the maker’s pages on 18 Nov 2026.

Is Text Generation Inference open source?

Yes. The code of Text Generation Inference is published at github.com/huggingface/text-generation-inference under the Apache-2.0 licence. Read on the maker’s pages on 18 Nov 2026.

Is Text Generation Inference still available?

Yes, as of 18 Nov 2026: its maker’s pages were read that day and carried no notice that Text Generation Inference has ended.

What changed recently in Text Generation Inference?

The latest entry in its maker’s changelog: 19 Dec 2025, “Version 3.3.7”. Before it: 17 Sept 2025, “Version 3.3.6”. Changelog checked on 16 Nov 2026.

Sources

Not confirmed

Prices
the official pricing page lists Hugging Face hosted compute, not TGI software tiers; no TGI-specific price was found.
Free plan
no TGI subscription or hosted-service tier structure was found.
Official website