llama.cpp is a highly optimized C/C++ library for running LLMs locally. You can install it on Windows, macOS, Linux, and Termux (Android) by running the official one-liner script curl -LsSf https://llama.app/install.sh | sh. This script detects your hardware and downloads the correct precompiled binaries automatically.
This tool runs models locally with minimal setup, providing a fast and private way to execute LLMs like Qwen, Llama, and Gemma on consumer hardware.
Supported Platforms and Hardware Acceleration
The installer script automatically detects your environment and sets up the correct build. Here is a summary of the supported platforms and their default acceleration backends:
| Platform | Recommended Environment | Acceleration Backend |
|---|---|---|
| macOS | Native Terminal | Metal (Apple Silicon M1/M2/M3/M4/M5) |
| Linux | bash / zsh | CUDA (Nvidia GPUs), ROCm (AMD), or CPU / Vulkan |
| Windows | PowerShell / WSL2 / Git Bash | CUDA, Vulkan, or CPU |
| Termux | Android App | CPU (ARM64 Neon optimized) |
According to community benchmarks, hardware acceleration (like Metal on Apple Silicon or CUDA on Nvidia GPUs) yields up to a 6x to 10x throughput improvement compared to CPU-only execution.
Step-by-Step Installation Guide
To install llama.cpp on your machine, open your terminal application and execute the installation script.
1. Run the Unified Installer Script
Execute the following one-liner command in your shell:
curl -LsSf https://llama.app/install.sh | sh(For Windows users running native PowerShell without curl/sh, install Git for Windows to run Git Bash, or run the command inside WSL2).
2. Verify Hardware Detection
The script runs a diagnostics check to determine your CPU architecture (such as x86_64 or ARM64) and looks for GPU acceleration libraries (like Metal, CUDA, ROCm, or Vulkan). It then downloads the precompiled binaries matching your system.
3. Update Your Shell PATH
Once the binary is downloaded, the script places it at ~/.llama-app/llama and copies it into ~/.local/bin/llama. If your system PATH does not include ~/.local/bin, add it by running:
export PATH="$HOME/.local/bin:$PATH"Restart your terminal session or run source ~/.zprofile (or source ~/.bashrc) to apply the changes.
Windows Specific Setup (Native vs WSL2)
On Windows machines, you have two primary options:
- WSL2 (Recommended for Nvidia CUDA): Open Ubuntu on WSL2 and run
curl -LsSf https://llama.app/install.sh | sh. WSL2 directly passes through Nvidia GPU drivers with zero overhead. - Direct Windows Binary: Download the pre-built
llama-bXXXX-bin-win-cuda-cu12.x-x64.zipdirectly from the llama.cpp GitHub Releases, unzip it, and add the folder to your Windows System Environment PATH.
Release Builds vs. Compiling from Source
The llama.app script installs the latest stable release of llama.cpp. This is the recommended option for most users because it is thoroughly tested.
If you need the absolute latest commits, custom GPU configurations, or mobile compilation:
- For Android: See my dedicated guide on How to Run LLM Models on Old Android Devices Locally to compile llama.cpp directly in Termux.
- For one-line mobile server: Check out How to Install and Run Ollama in Termux on Android.
Running Your First Model
Once installed via the script, you can use the unified llama binary to run local models.
First, download any model in the GGUF format (e.g. from Hugging Face or Unsloth). Then start a local, OpenAI-compliant API server by running:
llama serve -m path/to/model.gguf --port 8080You can now connect any front-end (like Open-WebUI, Chatbox, or AnythingLLM) to http://localhost:8080/v1.
馃挰 Frequently Asked Questions (FAQ)
How do I install llama.cpp on Windows?
On Windows, you can install llama.cpp via WSL2, Git Bash, or PowerShell by running curl -LsSf https://llama.app/install.sh | sh, or download pre-compiled release binaries with CUDA/Vulkan support from the official GitHub releases page.
Can I run llama.cpp without a dedicated GPU?
Yes. llama.cpp is highly optimized for CPUs using AVX2, AVX-512, and ARM Neon SIMD instruction sets, allowing smooth inference on standard laptops, mini PCs, and smartphones.
Does the llama.app installer script detect GPU acceleration automatically?
Yes. The installation script automatically detects Metal on macOS, CUDA on Linux/Windows, and Vulkan/ROCm on compatible devices, downloading the binary precompiled with the appropriate acceleration backend.
What is the difference between the llama.app binary and compiling from source?
The llama.app script provides a unified llama CLI with pre-compiled stable binaries for fast setup. Compiling from source gives you the granular llama-cli and llama-server executables with custom compilation flags.
