LLM Latency Tracker
Independent, provider-neutral latency & uptime for AI inference APIs — measured, not scraped.
🌐 Live: llmlatency.dev · 📊 JSON API · 🤖 MCP server · 🗓️ Deprecation calendar
Most "AI API latency" numbers come from the providers themselves, or from a benchmark run once and never updated. This project measures it continuously, from multiple regions, and publishes the result as an open dataset.
- Edge latency — full DNS → TCP → TLS → time-to-first-byte, measured with the Python standard library (no API key required).
- Inference latency — real time-to-first-token via a streaming request (optional, needs a provider key).
- Uptime — success rate per provider, per region.
- Regions — Europe (Germany), US (Central), Asia (Tokyo), South America (São Paulo). More welcome.
- ~45 providers — OpenAI, Anthropic, Google, Mistral, DeepSeek, xAI, Groq, Together, Fireworks, Cerebras, OpenRouter, Perplexity, plus Chinese models (GLM/Zhipu, Kimi/Moonshot, Qwen, MiniMax) and many more.
- Deprecation calendar — upcoming model retirements + migration targets, verified from official provider docs.
The site is a self-updating static site (Cloudflare Pages). The value isn't the code — it's the continuously-accumulated, distributed measurement archive. The code is open so the methodology is transparent.
For developers
curl https://llmlatency.dev/api/rankings.json
For AI agents
There's a real MCP server (Streamable HTTP) exposing a get_ai_api_latency tool backed by the live data:
curl -X POST https://llmlatency.dev/mcp \
-H 'Content-Type: application/json' \
-d '{"jsonrpc":"2.0","id":1,"method":"tools/call",
"params":{"name":"get_ai_api_latency","arguments":{"region":"eu-hetzner"}}}'
Also available: an MCP Server Card (/.well-known/mcp/server-card.json), a browser WebMCP tool, an API catalog (RFC 9727) and an Agent Skills index. Regions: eu-hetzner, us-central, ap-tokyo, sa-east (omit for all).
How it works
config.py — registry of providers + this node's REGION (env)
probe.py — network probe (DNS→TCP→TLS→TTFB, stdlib, no key) + inference probe (TTFT, needs key)
run.py — one probe cycle across all providers (run on a schedule)
db.py — SQLite time-series (the accumulated measurement archive)
aggregate.py — measurements → p50 / p95 / uptime rankings per region & provider
sitegen.py — rankings → static site (JSON API, OpenAPI, llms.txt, schema.org, MCP surface)
ingest.py — central endpoint that collects measurements from remote probe nodes
ship.py — probe node → central node shipper (watermark-based, never loses data on outage)
deprecations.py — model deprecation/migration calendar (only verified, sourced entries)
Each probe node runs with its own REGION, measures every provider, and writes to the time-series. For multi-region, remote nodes ship their measurements to a central node that aggregates and builds the site.
Run it yourself (no keys needed)
git clone https://github.com/mazamaka/llm-latency-tracker
cd llm-latency-tracker
REGION=local python3 run.py
python3 aggregate.py --region local
Runs on plain Python 3.12+ (standard library). httpx / loguru are optional.
Inference probes (real TTFT):
cp .env.example .env
pip install -r requirements.txt
REGION=local python3 run.py
python3 aggregate.py --region local --type inference
Build the site locally:
BASE_URL=https://example.com python3 sitegen.py
python3 -m pytest -q
See deploy/ for a container + a generic multi-region deployment guide.
Contributing
Especially welcome:
- New providers — add a
Provider(...) entry in config.py (host + public models endpoint is enough for edge probes).
- New regions — spin up a probe node in a new location and ship to a central node.
- Fixes & tests — CI runs
pytest + ruff on every push.
See CONTRIBUTING.md for dev setup, how to add a provider/region, and PR guidelines. Please keep the project's principle: measured, not scraped, and honest about the dataset's age.
License