🦀 llama.cpp VRAM Calculator

Estimate VRAM for a Hugging Face GGUF model: quant size, context, KV cache options, YaRN context extension, MTP heads, and multi-GPU split. Architecture is auto-fetched from the GGUF header (range-read — no full model download).

1. Model source

HF repo id

HF token (optional, for gated/private repos)

GGUF file

mmproj file (optional, multimodal)

mmproj weight bytes (auto from header; edit if needed)

Architecture (editable)

name

architecture

n_layer

n_embd

n_head

n_head_kv

training_ctx

params

rope_freq_base

n_expert (MoE)

n_expert_used

n_mtp

head_dim (0 = n_embd/n_head)

n_full_attn_layers (0 = all carry KV)

2. Inference options

Quantization

Target context (n_ctx)

KV cache dtype

Compute dtype

Flash attention

n_batch (-b, logits buffer)

n_ubatch (-ub, compute tile)

Expected prompt length (prefill only — does NOT affect VRAM)

MTP draft KV dtype

3. Multi-GPU budget

Per-GPU VRAM (GB, comma-separated; suffix 'u' = unified)

split-mode

main-gpu (none mode)

--tensor-split ratios (optional, comma-sep)

Safety margin %

--parallel (server slots)

Size KV for N slots × C ctx (scales VRAM)

Live: every slider/dropdown above updates the results and graphs on change. The Compute button is a manual trigger for the same recompute.

Results

VRAM vs context (by quant)

VRAM vs quant @ ctx (color = quality)

VRAM scratch vs n_batch

Command (copyable)

Assumed on by llama-server: continuous batching, jinja chat templates, prompt caching (--cache-prompt), flash-attention auto. The command notes these rather than emitting flags.

Auto-fit solvers

Uses the current architecture + GPU budget. Run the calculator first (or load a preset/fetch a GGUF) so the architecture is populated, then pick a solver.

Quant for max-context solver

Context for best-quant solver

Context for min-GPU solver

Quant for min-GPU solver