FitLLM

Official3 toolsLive

by click6067-ship-it · JavaScript

Will this LLM fit on your GPU, multi-GPU rig or Mac? Exact VRAM & KV-cache math. Read-only.

Topics

apple-siliconkv-cachellmmemory-calculatormoequantizationvramggufinferencellama-cpplocal-llmlocalllamamlxnvidiaollamavram-calculatoramdclimlawill-it-run

Captured live from the server via tools/list.

check_llm_fit

Check whether a specific local LLM fits in the memory of a specific GPU or Apple Silicon Mac. Returns fits/tight/won't-fit verdict with the full memory breakdown (weights, KV cache, overhead), max context, and a concrete fix if it doesn't fit. Use this whenever a user asks anything like "can I run <model> on my <GPU/Mac>?", "will <model> fit in <N>GB?", or "what do I need to run <model>?". Architecture-aware math (MLA, sliding-window, hybrid attention, MoE) — more accurate than rule-of-thumb estimates.

Parameters7

model	string	required	LLM name, fuzzy — e.g. "GLM-4.7-Flash", "gpt-oss-20b", "gemma 31b"
gpu	string	optional	GPU name, fuzzy — e.g. "RTX 4090", "RX 7900 XTX", "A100 80GB". Multi-GPU rigs: join with + — e.g. "RTX 5090 + RTX 3090" (VRAM pools across cards). Provide gpu OR mac_ram_gb.
gpu_count	integer	optional	Number of identical copies of the gpu (e.g. gpu="RTX 3090", gpu_count=2 for a 2×3090 rig). Default 1.
mac_ram_gb	integer	optional	Apple Silicon unified memory in GB — e.g. 16, 64, 512. Provide gpu OR mac_ram_gb.
quant	string	optional	Weight quantization. GPU: Q4_K_M(default)/Q5_K_M/Q6_K/Q8_0/FP16. Mac: 4/8(default)/16 (bits).
context_tokens	integer	optional	Context length in tokens (default 8192)
kv_bits	number	optional	KV-cache quantization bits (default 16 = F16)

Raw schema

{
  "type": "object",
  "properties": {
    "model": {
      "type": "string",
      "description": "LLM name, fuzzy — e.g. \"GLM-4.7-Flash\", \"gpt-oss-20b\", \"gemma 31b\""
    },
    "gpu": {
      "type": "string",
      "description": "GPU name, fuzzy — e.g. \"RTX 4090\", \"RX 7900 XTX\", \"A100 80GB\". Multi-GPU rigs: join with + — e.g. \"RTX 5090 + RTX 3090\" (VRAM pools across cards). Provide gpu OR mac_ram_gb."
    },
    "gpu_count": {
      "type": "integer",
      "minimum": 1,
      "maximum": 8,
      "description": "Number of identical copies of the gpu (e.g. gpu=\"RTX 3090\", gpu_count=2 for a 2×3090 rig). Default 1."
    },
    "mac_ram_gb": {
      "type": "integer",
      "minimum": 8,
      "maximum": 2048,
      "description": "Apple Silicon unified memory in GB — e.g. 16, 64, 512. Provide gpu OR mac_ram_gb."
    },
    "quant": {
      "type": "string",
      "description": "Weight quantization. GPU: Q4_K_M(default)/Q5_K_M/Q6_K/Q8_0/FP16. Mac: 4/8(default)/16 (bits)."
    },
    "context_tokens": {
      "type": "integer",
      "minimum": 1024,
      "description": "Context length in tokens (default 8192)"
    },
    "kv_bits": {
      "type": "number",
      "enum": [
        16,
        8,
        4
      ],
      "description": "KV-cache quantization bits (default 16 = F16)"
    }
  },
  "required": [
    "model"
  ],
  "additionalProperties": false,
  "$schema": "http://json-schema.org/draft-07/schema#"
}

what_fits_on_hardware

Rank which popular local LLMs fit on a given GPU or Apple Silicon Mac (at ~4-bit quantization, 8K context) — models that fit come first, biggest first, with max context each. Use when a user asks "what can I run on my <GPU/Mac/N GB>?", "best local model for my machine?", or gives hardware without naming a model.

Parameters3

gpu	string	optional	GPU name, fuzzy. Multi-GPU rigs: join with + (e.g. "RTX 5090 + RTX 3090"). Provide gpu OR mac_ram_gb.
gpu_count	integer	optional	Number of identical copies of the gpu. Default 1.
mac_ram_gb	integer	optional	Apple Silicon unified memory GB. Provide gpu OR mac_ram_gb.

Raw schema

{
  "type": "object",
  "properties": {
    "gpu": {
      "type": "string",
      "description": "GPU name, fuzzy. Multi-GPU rigs: join with + (e.g. \"RTX 5090 + RTX 3090\"). Provide gpu OR mac_ram_gb."
    },
    "gpu_count": {
      "type": "integer",
      "minimum": 1,
      "maximum": 8,
      "description": "Number of identical copies of the gpu. Default 1."
    },
    "mac_ram_gb": {
      "type": "integer",
      "minimum": 8,
      "maximum": 2048,
      "description": "Apple Silicon unified memory GB. Provide gpu OR mac_ram_gb."
    }
  },
  "additionalProperties": false,
  "$schema": "http://json-schema.org/draft-07/schema#"
}

list_supported

List the built-in model names and hardware names this fit-checker knows (for mapping user wording to exact names). Any public HuggingFace model also works via fitllm.run.

Parameters

No parameters.

Raw schema

{
  "type": "object",
  "properties": {},
  "$schema": "http://json-schema.org/draft-07/schema#"
}

README.md11 KBView on GitHub

FitLLM Engine

npx fitllm — one-line fit verdict with the full memory breakdown

Live: https://fitllm.run · Bilingual · Free · No ads · No login

Open engine: fitllm-engine (MIT · npm fitllm-engine · npx fitllm)

Zero dependencies. One readable file: engine.js. Conformance-vector tested. MIT.

bash

npx fitllm "GLM-4.7-Flash" --gpu 4090     # ✓ FITS — 21.9/24 GB, free 2.1 GB
npx fitllm "gpt-oss-120b" --mac 64        # ✗ WON'T FIT → what to change to make it fit
npx fitllm "Qwen 3.6 35B" --gpu "5090 + 3090"   # multi-GPU rig — VRAM pools (56GB), even mixed cards
npx fitllm --detect                       # reads this machine's real hardware

Why a CLI? The "will it run?" question is born in the terminal — one line before ollama pull. No install, no tab-switching, and it reads your actual hardware with --detect instead of asking you to know your VRAM. Exit code 0/1 makes it a pre-download guard:

bash

# in your model-pull script — stop BEFORE the 40 GB download:
npx fitllm "gpt-oss-120b" --detect || { echo "won't fit — aborting pull"; exit 1; }

This is the open calculation core of FitLLM. The math is open so you can audit it.

Ask an LLM "does Qwen 3.6 fit my GPU?" and it pattern-matches to an architecture from its training cutoff — and usually says no. Catalog-based calculators lag new releases. FitLLM reads each model's official config.json live, so it's right on day-one releases and on the hybrid / sliding-window / MoE architectures that naive formulas get wrong.

Covers Apple Silicon unified memory (M1–M5, Pro/Max/Ultra — up to the 512GB Mac Studio), NVIDIA GPUs (RTX 20/30/40/50, workstation RTX 6000 Ada / RTX PRO 6000, datacenter A100/H100/H200/B200), AMD Radeon (RX 7000/9000, PRO W7900) and multi-GPU presets (2×3090, 2×4090, 4×3090) — with GGUF Q-tier weight quantization kept separate from KV-cache quantization. Every hardware number is cross-verified against ≥2 independent sources (source URLs embedded per-value in engine.js).

Why most LLM memory calculators are wrong

Almost every "can I run this LLM?" calculator estimates the KV cache with the textbook formula:

code

KV ≈ 2 × num_layers × num_kv_heads × head_dim × context_length × bytes

That assumes every layer keeps a full-context KV cache with one uniform head shape. True for Llama-1/2 — wrong for most 2025–2026 models:

Model	What naive formulas miss	Naive KV	FitLLM KV	Off by
Gemma 4 31B @131K, 8-bit	50 of 60 layers are sliding-window (keep only the last 1024 tokens); the 10 global layers use a different head shape (4 KV-heads × 512, not 16 × 256)	~60 GB	~5.4 GB	11×
Qwen 3.6 27B @131K, 8-bit	48 of 64 layers are linear attention (Gated DeltaNet) — no growing KV cache	~16 GB	~4 GB	4×
GLM-4.7-Flash @128K, bf16	MLA: K/V compressed into one shared latent (512+64 dims, cached once — not per-head K and V)	~117 GB	~6.6 GB	17.8×
Plain dense (Llama, Mistral…)	nothing — standard transformer	same	same	1× ✅

An 11× error flips the verdict: a naive calculator says Gemma 4 31B won't fit in 64 GB at long context, when it fits comfortably.

The four things they ignore

Sliding-window attention (Gemma 2/3/4, gpt-oss): most layers only keep the last N tokens, so their KV stops growing. Only the global layers scale with full context.
Hybrid / linear attention (Qwen 3.6, many 2026 models): linear-attention layers use a fixed-size recurrent state, not a growing KV cache.
MLA — Multi-head Latent Attention (GLM-5.2, GLM-4.7-Flash, DeepSeek family): the cache is a single low-rank latent (kv_lora_rank + RoPE dims) shared across all heads — per-head "2 × heads × head_dim" formulas over-count by an order of magnitude. Verified against the DeepSeek-V2 paper (arXiv:2405.04434) and the official DeepSeek-V3 inference code.
Heterogeneous head dims + MoE: global layers can use a different head_dim (Gemma 4: 512 vs 256). MoE keeps every expert in memory while activating only a few per token.

This engine models each layer type separately, verified against official HuggingFace config.json files.

What it computes

code

Total = Parameters (quantization-adjusted)
      + KV cache (per layer kind: sliding / global / linear / dense)
      + Runtime overhead (quant metadata + KV block padding + activations + fixed)
      + macOS base (Apple Silicon unified memory)

Plus decode-speed estimate (bandwidth ÷ active-params) and an parseHfConfig() that turns any HuggingFace config into the model shape above.

Usage

import { simulate, LOCAL_MODELS, estimateSpeed, parseHfConfig } from './engine.js';

const model = LOCAL_MODELS.find((m) => m.name === 'Gemma 4 31b');
const sim = simulate(model, /*ram*/ 64, /*ctx*/ 131072, /*bits*/ 8);
// → { used, free, verdict: 'yes'|'tight'|'no', param, kv, rt, os, maxContext, ... }

estimateSpeed(model, 'M5 Max', 8, /*gpuCores*/ 40); // ≈ tok/s

// any HuggingFace model:
const m = parseHfConfig('Qwen/Qwen3-32B', configJson, totalSizeBytes);

Verification

Architecture values checked against official HuggingFace config.json.
Gemma 4 31B full-context KV reproduces 20.78 GiB, matching the published architecture analysis. Reproduce it by hand:

code

global: 10 layers × 2(K,V) × 4 heads × 512 dim × 2 B × 262,144 = 21,474,836,480 B
local:  50 layers × 2(K,V) × 16 heads × 256 dim × 2 B × 1,024  =    838,860,800 B
total = 22,313,697,280 B ÷ 1024³ = 20.78 GiB

Calibration: Qwen 3.6 35B-A3B @128K, 8-bit ≈ 54 GB (matches real local runs).
MLA per-token cost: GLM-4.7-Flash = (512 + 64) × 2 B × 47 layers = 54,144 B/token — pinned by conformance vectors.

All figures are estimates — real usage varies with the runtime (MLX/Ollama/llama.cpp), OS state, and quantization scheme.

Conformance vectors

vectors/fit-vectors-v1.json pins 14 language-neutral test vectors (exact KV bytes, per-token costs, fit verdicts) derived by hand from official config.json values — e.g. "Gemma 4 31B at 262,144 ctx, bf16 = exactly 22,313,697,280 bytes". Any implementation in any language conforms if every vector passes — run ours with node vectors/run.mjs.

Why this matters: the formulas are easy to copy; a verified answer key is not. If you port this engine to Python, Rust or Go, you don't become an untrusted fork — pass the vectors and you're a conformant implementation of the same standard. Port the engine, keep the vectors.

The Fit Census — every model × every device, one truth table

census/ holds 6,000+ verdicts (19 models incl. draft tier × 88 GPUs/Macs × quant tiers) computed by this engine — as CSV/JSON you can import, chart or cite, plus a starter matrix ("biggest model that fits comfortably per device"). Regenerate it yourself: npm run census. Real-world measurements land next to predictions via fixtures/ PRs — predicted vs. measured, in public.

Embed a fit badge

Show whether a model runs on given hardware — live from the engine, one line in any README or model card:

markdown

![fits](https://img.shields.io/endpoint?url=https%3A%2F%2Ffitllm.run%2Fapi%2Fbadge%3Fmodel%3DGLM-4.7-Flash%26gpu%3D4090)

fits

Params: model (name, fuzzy), gpu (name, fuzzy) or ram (GB, Apple unified memory), optional quant (GGUF tier / 4|8|16), ctx, kv. Verdict color: green fits · yellow tight · red won't fit.

Why embed it? The #1 question under every model card and local-AI tutorial is "will it run on my machine?" The badge answers it live from the engine — recomputed when the data updates, not a stale claim frozen into your README. If you publish models or write guides: one line replaces a whole FAQ paragraph and cuts the "it OOM'd on my 8GB card" issues before they're filed.

Ask your AI assistant (MCP)

The engine runs as a public MCP server at https://fitllm.run/api/mcp — connect it once and your assistant answers "can I run X on my Y?" with this engine's math instead of guessing from stale training data (LLMs routinely get KV-cache math wrong — see the 17.8× table above).

Claude (web / desktop / mobile): Settings → Connectors → Add custom connector → paste https://fitllm.run/api/mcp
Claude Code: claude mcp add --transport http fitllm https://fitllm.run/api/mcp
Cursor / Windsurf: add to mcp.json → { "mcpServers": { "fitllm": { "url": "https://fitllm.run/api/mcp" } } }
ChatGPT: Settings → Apps → Advanced → Developer mode → add MCP server (Plus/Pro)

Tools: check_llm_fit (verdict + full memory breakdown + fix suggestion — supports multi-GPU rigs like "RTX 5090 + RTX 3090"), what_fits_on_hardware (ranked list for your machine), list_supported. Resources: fitllm://models, fitllm://hardware, fitllm://census, fitllm://engine. Intentionally open: read-only, stateless, no auth, no secrets — every call is a pure function of public data.

Listed on: official MCP registry (run.fitllm/fitllm) · Glama · mcp.so · Smithery

For agents & scripts — plain HTTP API

No MCP client? One GET, no auth, no key — JSON by default, plain text for curl:

bash

curl 'https://fitllm.run/api/check?model=gemma%204%2031b&gpu=4090'
# multi-GPU rigs: gpu=5090%2B3090 · Mac: ram=64 · usage: curl https://fitllm.run/api/check

Open data: the full Fit Census (6,000+ verdicts, CC0) at fitllm.run/data and on Hugging Face Datasets. Try the engine in-browser: HF Space demo.

Principles

No ads. No login. No affiliate links. Output is never for sale. Fit is a winnable, verifiable claim; raw tok/s is not — so this engine refuses speed predictions rather than dress a guess as precision.

Help calibrate

Ran a model and measured real peak memory? Report a measurement — it improves the estimates for everyone.

Built by

yongha — GitHub. Powers fitllm.run.

License

Install

Tools (3)

check_llm_fitwhat_fits_on_hardwarelist_supported

Remote endpoint

Streamable HTTP

Hosted server - connect over the network, no local install.

url

https://fitllm.run/api/mcp

Add to VS Code Add to Cursor

claude_desktop_config.json

json

{
  "mcpServers": {
    "fitllm": {
      "command": "npx",
      "args": [
        "-y",
        "mcp-remote",
        "https://fitllm.run/api/mcp"
      ]
    }
  }
}

Desktop config is stdio-only; this bridges via mcp-remote. Native remote: Settings > Connectors.