Zum Inhalt springen
AI.info

AI Tools

llama.cpp

llama.cpp: Preise, Fähigkeiten, Updates und Alternativen, von den Seiten des Anbieters gelesen.

Die Website von llama.cpp

Auf Englisch

Key facts

Price
Not stated on the maker’s pages
Free plan
Yes Read on github.com
Platforms
macOS · Windows · Linux · iOS · Android · CLI · API · Web Read on github.com
API
Yes Read on github.com
Open source
Yes · MIT Read on github.com
Latest version
b12557 Read on github.com
Last update
19 Nov 2026 Read on github.com

llama.cpp is an open-source C/C++ runtime for running language and vision-language models on local machines or in cloud environments. It supports quantized models and CPU, GPU, and hybrid inference across multiple hardware backends.

Developers can use its command-line tools or run a server with a built-in chat interface and compatible API routes. It is inference software, not a hosted model subscription; model downloads, hardware, and any model-specific terms are separate.

Category
Developer Tools
Read from the maker’s pages
2026-11-18

Company

ggml.ai

Features

  • MIT-licensed open-source C/C++ inference runtime
  • Runs F16 and quantized models on CPUs and GPUs
  • Supports multiple quantization levels, from 1.5-bit to 8-bit
  • Uses hardware backends including CUDA, HIP, Metal, Vulkan, and SYCL
  • Combines CPU and GPU inference for models exceeding available VRAM
  • Provides an OpenAI-compatible server API for chat and embeddings
  • Includes a built-in chat interface and multimodal support

Use cases

  • Run a downloaded language model on a personal computer
  • Serve local models to applications through compatible API routes
  • Test quantized models on CPU- or GPU-equipped hardware
  • Build a local chat interface around an inference server
  • Deploy model inference across supported hardware backends

Pricing

Free

Free

  • Unlimited public/private repositories
  • Dependabot security and version updates
  • 2,000 CI/CD minutes/month
  • 500MB of Packages storage
  • Issues & Projects
  • Community support

Team

$4/user/mo

  • Access to GitHub Codespaces
  • Repository rules
  • Multiple reviewers in pull requests
  • Draft pull requests
  • Code owners
  • Required reviewers

Enterprise

$21/user/mo

  • GitHub Copilot access
  • Agent governance
  • Data residency
  • Enterprise Managed Users
  • User provisioning through SCIM
  • Enterprise Account to centrally manage multiple organizations

Capabilities

  • Command line — “llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF” source
  • Choice of models — “Download and run a model directly from Hugging Face” source
  • Self-hosted — “The main goal of llama.cpp is to enable LLM (and VLM) inference with minimal setup and state-of-the-art performance on a wide range of hardware - locally and in the cloud.” source
  • API — “Launch OpenAI-compatible API server” source

Latest updates

Where it works

macOS · Windows · Linux · iOS · Android · CLI · API · Web

Alternatives to llama.cpp

  • vLLM

    vLLM runs and serves large language models on supported hardware, with APIs for applications to send inference requests.

  • LocalAI

    LocalAI runs open AI models on your own hardware, with a web interface and APIs for chat, images, speech, video, and more.

  • Ollama

    Ollama runs open AI models locally or in Ollama’s cloud, with a CLI and API for using them in apps and coding tools.

  • Text Generation Inference

    Text Generation Inference is an open-source toolkit for deploying and serving large language models through an API.

  • Unsloth

    Free, open-source desktop software for running and training AI models locally, with tools for image generation, web search and coding agents.

  • Hermes Agent

    Hermes Agent is an open-source agent that runs tasks, uses tools, and builds persistent memory and reusable skills.

More from ggml.ai

  • whisper.cpp

    A C/C++ implementation for running Whisper speech recognition locally, with command-line, library, server, and WebAssembly examples.

Questions about llama.cpp

Is llama.cpp free?

Yes. The maker’s pages state that llama.cpp can be used for free. Read on the maker’s pages on 18 Nov 2026.

Does llama.cpp have an API?

Yes. The maker’s pages state that llama.cpp has an API. Read on the maker’s pages on 18 Nov 2026.

Which platforms does llama.cpp run on?

llama.cpp runs on macOS, Windows, Linux, iOS, Android, CLI, API and Web, as its maker’s pages state. Read on the maker’s pages on 18 Nov 2026.

Is llama.cpp open source?

Yes. The code of llama.cpp is published at github.com/ggml-org/llama.cpp under the MIT licence. Read on the maker’s pages on 18 Nov 2026.

Is llama.cpp still available?

Yes, as of 18 Nov 2026: its maker’s pages were read that day and carried no notice that llama.cpp has ended.

What changed recently in llama.cpp?

The latest entry in its maker’s changelog: 19 Nov 2026, “b12557: CUDA: implement support for duplicate expert IDs (#32505)”. Before it: 19 Nov 2026, “Version b12556”. Changelog checked on 20 Nov 2026.

Sources

Not confirmed

Prices
No official pricing page was found in the pricing search; pricing_details is empty.
Official website