CheckMCP
Vendor-neutral quality / security / context-cost audit & score for any MCP server.
One uvx/pipx command → an MCP Score /100 + causal opportunities (why the score), Lighthouse-style.
Installed from PyPI as audit-mcp (the name checkmcp was already taken); the command is audit-mcp. Brand, site and repo remain CheckMCP / checkmcp.dev.
uvx audit-mcp https://mcp.deepwiki.com/mcp
pipx run audit-mcp https://mcp.context7.com/mcp --json
audit-mcp https://my-mcp.example.com/mcp --token "$TOKEN"
Use it as an MCP server
audit-mcp mcp turns the auditor itself into an MCP server (stdio) exposing one tool,
audit_mcp_server — so your agent can answer "is this MCP server safe?" mid-conversation.
claude mcp add audit-mcp -- uvx audit-mcp mcp
{ "mcpServers": { "audit-mcp": { "command": "uvx", "args": ["audit-mcp", "mcp"] } } }
Registry name: io.github.H129hj/checkmcp
No dependencies (stdlib only). tiktoken optional for exact token counts.
What it measures (7 pillars)
- Security — OWASP MCP Top 10 (tool poisoning, hardcoded secrets, command injection), lethal-trifecta.
- Tool design — sprawl/consolidation (percentile-calibrated on real servers: median ~7 tools, p95 ~42).
- Schemas / desc — descriptions +
inputSchema/outputSchema completeness.
- Context-cost — tokens spent on
tools/list, paid on every request (the #1 pain of 2026).
- Compliance — protocol-version gap, annotations, JSON-RPC error conformance, OAuth discovery.
- Reliability — single-shot today (not credited; continuous T3 monitoring on checkmcp.dev).
- Coverage — the 3 primitives (tools + resources + prompts).
Hard floors: secret-in-schema → cap D, failed handshake → cap F. Every penalty is attributed: measure → mechanism → effect → Δscore.
CLI flags
| flag | what |
|---|
--json | machine-readable report |
--badge | SVG badge + README embed snippets |
--html | standalone SEO/GEO page (JSON-LD SoftwareApplication + FAQ) |
--repo owner/name | add maintenance/license/provenance signal from GitHub |
--token <bearer> | audit an OAuth-protected server |
--min-score N | CI: exit 1 if MCP Score < N |
--baseline file | CI: pin tool definitions; fail on regression (rug-pull) |
--gh-summary | CI: write a Markdown summary to $GITHUB_STEP_SUMMARY |
--deep | runtime depth via an external scanner (mcp-scan/snyk) if present |
--evals | behavioral sandbox: actually invokes read-only tools with canary inputs to catch tool-output prompt-injection, exfiltration vectors, secret/PII leakage and context bombs (sends real traffic; CI-fails on a malicious verdict) |
GitHub Action
name: MCP audit
on: [push, pull_request]
jobs:
checkmcp:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: H129hj/checkmcp@v1
with:
url: https://my-mcp.example.com/mcp
min-score: "70"
baseline: .checkmcp-baseline.json
Behavioral evals (--evals)
Static analysis catches declared danger; --evals catches runtime danger by actually invoking
read-only tools with canary inputs and inspecting the responses for tool-output prompt-injection,
exfiltration vectors and secret/PII leakage (multilingual; optional callback-canary confirms exfil).
CI-fails on a malicious verdict.
Self-hosted security gateway
Beyond auditing, CheckMCP ships an in-band MCP gateway — a proxy you put between your agent and an
MCP server. It inspects every call, and in active mode blocks/strips tool-poisoning & exfiltration
before they reach the agent. Run it in your own infra (tool traffic never leaves your network):
docker pull ghcr.io/h129hj/checkmcp-gateway:latest
docker run -p 8080:8080 -e GATEWAY_BACKEND_URL=https://mcp.example.com/mcp \
-e GATEWAY_MODE=active -e GATEWAY_SECRET=$(openssl rand -hex 16) \
ghcr.io/h129hj/checkmcp-gateway:latest
See GATEWAY.md for config (passive/active, OAuth backends, policy, logs).
Hosted
Full reports, public directory, live badges, continuous drift monitoring, a governance policy API and a
hosted gateway at checkmcp.dev.
Honest limitations
- Percentile bands come from a growing corpus (one+ registries) — widening over time.
- Exact tokens with
pipx install "audit-mcp[exact-tokens]" (cl100k_base); otherwise chars/4 approximation.
- Pillar weights are expert priors.
python -m checkmcp.calibrate samples.json validates them against a labeled agent-success sample (per-pillar correlation + OLS-suggested weights + construct-validity R²) — supply real outcomes to close the loop.
MIT.