Tools
llama.cpp vs Text Generation Inference
llama.cpp or Text Generation Inference? Their plans monthly and yearly, the capabilities their makers state, security and latest updates, side by side — read from the makers' own pages.
In inglese
In short
- Both: Command line, Choice of models, Self-hosted, API
- Only Text Generation Inference states: Runs models for you, Traces and evaluates
llama.cpp
Open-source C/C++ software for running large language and vision-language models locally or in the cloud.
Plans
Free
Free
- Unlimited public/private repositories
- Dependabot security and version updates
- 2,000 CI/CD minutes/month
- 500MB of Packages storage
- Issues & Projects
- Community support
Team
- $4 per user/month
- Access to GitHub Codespaces
- Repository rules
- Multiple reviewers in pull requests
- Draft pull requests
- Code owners
- Required reviewers
Enterprise
- Starting at $21 per user/month
- Data residency
- Enterprise Managed Users
- User provisioning through SCIM
- Enterprise Account to centrally manage multiple organizations
- Environment protection rules
- Repository rules
Prices checked 2026-09-24 on the maker’s page.
Capabilities
- Command line — “llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF” source
- Choice of models — “llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF” source
- Self-hosted — “locally and in the cloud.” source
- API — “Launch OpenAI-compatible API server” source
Latest updates
- b11160
Adds Vulkan int8 cooperative-matrix quantized matmul for AMD RDNA3 and RDNA4, including q8_0, q4_1, q5_0, and q5_1.
- b11159
Handles misalignment in conv_2d and conv_3d.
- b11158
Text Generation Inference
Open-source server for deploying and serving language models with streaming, batching, quantization, and multi-GPU inference.
Plans
Free
Free
- Unlimited public/private repositories
- Dependabot security and version updates
- 2,000 CI/CD minutes/month
- 500MB of Packages storage
- Issues & Projects
- Community support
Team
- $4 USD per user/month
- Access to GitHub Codespaces
- Repository rules
- Multiple reviewers in pull requests
- Draft pull requests
- Code owners
- 3,000 CI/CD minutes/month
Enterprise
- Starting at $21 USD per user/month
- Data residency
- Enterprise Managed Users
- User provisioning through SCIM
- Enterprise account to centrally manage multiple organizations
- Environment protection rules
- Audit Log API
Prices checked 2026-09-25 on the maker’s page.
Capabilities
- Command line — “To see all options to serve your models (in the code or in the cli): text-generation-launcher --help” source
- Choice of models — “text-generation-launcher --model-id mistralai/Mistral-7B-Instruct-v0.2” source
- Self-hosted — “You can also opt to install text-generation-inference locally.” source
- API — “You can consult the OpenAPI documentation of the text-generation-inference REST API using the /docs route.” source
- Runs models for you — “Text Generation Inference (TGI) is a toolkit for deploying and serving Large Language Models (LLMs).” source
- Traces and evaluates — “Production ready (distributed tracing with Open Telemetry, Prometheus metrics)” source
Latest updates
- v3.3.7 (v3.3.7)
Added support for max_image_fetch_size and announced maintenance mode.
- v3.3.5 (v3.3.5)
Added Gemma 3 sliding-window support and XPU LoRA support.
- v3.3.4 (v3.3.4)
Added initial Gaudi support for Gemma 3 text and VLM models.