io.github.Incultnitollc/mcp-probe

Official

by Incultnitollc · TypeScript

Inspect and score any MCP server 0-100 on publishability, schema and protocol before publish.

Inspect and score any MCP server 0-100 on publishability, schema and protocol before publish. The tool analyzes every tool, resource, and prompt exposed by the server and generates a health report with a pass/fail scorecard, built on the Anthropic Model Context Protocol (MCP) spec.

🛠️ Key Features

Diagnoses MCP server health via a single command
Evaluates publishability, schema correctness, and protocol conformance
Generates a structured pass/fail scorecard and health report
CLI-based tool published under npm scope @incultnitollc/mcp-probe
Based on the Model Context Protocol (MCP) specification

🚀 Use Cases

Pre-publish validation of MCP servers
Automated quality checks for tools, resources, and prompts
Compliance testing against MCP schema and protocol expectations

⚡ Developer Benefits

Clear, actionable diagnostics for MCP server readiness
Standardized health reporting aligned with MCP spec
Lightweight CLI for quick integration into CI/CD
Open-source tooling with reproducible testing workflow

⚠️ Limitations

Readme excerpt indicates scope and naming constraints; verify version history and npm scope details
Subject to upstream MCP spec changes and tool availability within the repository

Topics

anthropicclaudeclideveloper-toolsmcpmodel-context-protocoltestingaillmmcp-servermcp-testingmcp-toolsmcp-validationvalidationvalidator

README.md12 KBView on GitHub

mcp-probe — one command to diagnose your MCP server

mcp-probe

One command to diagnose your MCP server.

Tests every tool, resource, and prompt your server exposes — then gives you a health report with a pass/fail scorecard.

Built on the Anthropic Model Context Protocol (MCP) spec.

Note: Published to npm as @incultnitollc/mcp-probe. The CLI binary is mcp-probe. The unscoped name mcp-doctor on npm is owned by an unrelated tool, so this project ships under a scope. Versions <= 0.2.1 shipped under the deprecated @incultnitostudiosllc scope — install @incultnitollc/mcp-probe instead.

mcp-probe demo

code

npx @incultnitollc/mcp-probe test "npx -y @modelcontextprotocol/server-everything"

Test your MCP server in 30 seconds

Check	Description
Tool calling	Calls every tool with auto-generated sample arguments based on the input schema
Resource reading	Reads every resource and verifies content is returned
Prompt rendering	Gets every prompt with sample arguments and verifies messages are returned
Schema validation	Checks tool schemas for missing descriptions, broken required fields, malformed types
Health scoring	Summarizes everything into a pass/fail scorecard

Install

bash

npm install -g @incultnitollc/mcp-probe

Or run directly:

bash

npx @incultnitollc/mcp-probe test "your-server-command"

Usage

Local stdio server

bash

npx @incultnitollc/mcp-probe test "npx -y @modelcontextprotocol/server-everything"

Remote server (Streamable HTTP)

bash

npx @incultnitollc/mcp-probe test https://your-server.example.com/mcp

Remote server (SSE)

bash

npx @incultnitollc/mcp-probe test https://your-server.example.com/mcp --transport sse

Authenticated remote server

bash

npx @incultnitollc/mcp-probe test https://your-server.example.com/mcp \
  --header "Authorization: Bearer $TOKEN"

Options

Flag	Description
`--json`	Output results as JSON
`--timeout <ms>`	Per-operation timeout (default 30000)
`--transport <kind>`	Force `stdio`, `sse`, or `http` (auto-detected from target)
`--header <Name: value>`	Add header to remote transport. Repeatable.

Exit codes

0 — All checks passed
1 — One or more checks failed (useful for CI gates)

JSON output

Use --json to get structured output for automation:

bash

mcp-probe test --json "your-server" | jq '.score'

json

{
  "toolsCallable": 12,
  "toolsTotal": 13,
  "resourcesReadable": 7,
  "resourcesTotal": 7,
  "promptsGettable": 3,
  "promptsTotal": 4,
  "schemaErrors": 0,
  "schemaWarnings": 1
}

How tool calling works

mcp-probe auto-generates arguments for each tool based on its inputSchema:

Only required fields get values (safest approach)
Uses default values and enum first choices when available
Infers smart defaults from field names (url → https://example.com, email → test@example.com)
Falls back to type-appropriate defaults (string → "test", number → 1, boolean → false)

This means tools with complex required inputs may fail — and that's useful information. It tells you your tool isn't self-contained enough for automated testing.

Publishability score (v1.1.0+)

mcp-probe ships a second, complementary check: a publishability composite that scores your server 0–100 on whether its schemas, descriptions, and metadata are ready for other people to install. Run it as a shorthand:

bash

npx @incultnitollc/mcp-probe score "npx -y @your-scope/your-server" --package ./package.json

Or fold it into a full test run with --publishability:

bash

npx @incultnitollc/mcp-probe test "npx -y @your-scope/your-server" --publishability --package ./package.json

The composite combines three sub-scores — Protocol (does the wire format work), Edge cases (does it handle weird inputs), and Publishability (would a stranger understand your tools) — and a five-axis breakdown across the publishability dimension:

Axis	What it checks
`description-five-axis`	Per-tool description density across purpose, mutation, side-effects, invariants, examples. Tools below 3.0/5 axes fire a ≤60 composite cap.
`enum-shape`	Catches prose-only enums (e.g. `"one of: open, closed"` in the description with no JSON Schema `enum`).
`mutation-legibility`	Does each tool tell a planner it mutates, or only reads? Name prefix / description signal / annotation all count.
`anti-purpose-clause`	High-blast tools (delete, send, transfer) should include a "do not use for X, prefer Y" pointer to a narrower tool.
`distribution-metadata`	npm package readiness — description length, keyword count, `repository` / `license` / `homepage` fields. Skipped without `--package`.

What scores look like on real servers

The five official Anthropic MCP servers all land at 60/100 under v1.1.0 — the description-five-axis cap fires on every one. That's not a bug in the rubric; that's the bar Anthropic ships at, and the bar most servers will start from. Full scorecards in docs/publishability-scorecards/.

CI gate

yaml

- uses: incultnitollc/mcp-probe@v1
  with:
    command: 'node dist/index.js'
    publishability: 'true'
    package: './package.json'
    fail-under: '70'

Pre-publish vs install-time

mcp-probe's publishability score is the pre-publish quality lane — for server authors before they ship. For the install-time security lane — server installers before they connect a third-party server — see @stephenywilson/mcp-doctor. Different audiences, complementary tools.

Contract testing — "VCR for MCP" (v1.2.0+)

test/score tell you if your server is healthy today. Contract testing tells you what changed since last time — so a breaking schema edit or a poisoned tool description never ships silently.

Record a snapshot of your server's contract (its tools, resources, prompts, and their schemas — no traffic, no side effects), commit the .mcpvcr file, then diff or gate every PR against it.

bash

# 1. Record a baseline (lists only — never calls a tool)
mcp-probe record "node dist/index.js" --out .mcp/contract.mcpvcr
git add .mcp/contract.mcpvcr && git commit -m "record MCP contract baseline"

# 2. See what a change did to the contract
mcp-probe diff --baseline .mcp/contract.mcpvcr "node dist/index.js"

# 3. Gate CI — exit 1 on breaking or security changes
mcp-probe gate --baseline .mcp/contract.mcpvcr "node dist/index.js"

Every change is classified so the gate is meaningful, not noisy:

Severity	Examples
❌ breaking	tool removed · new required argument · property removed · type changed · enum narrowed
⚠️ security	tool description mutated (rug-pull / tool-poisoning) · `readOnlyHint` dropped · tool became destructive
✅ additive	new tool · new optional field · enum widened
ℹ️ info	server name · resource/prompt description text

gate fails on breaking,security by default — tune with --fail-on breaking (or any comma-separated set). Snapshots are deterministic: re-recording an unchanged server produces a byte-identical file, so committed baselines and diffs stay clean. Compare two recorded files offline with --against <file> instead of a live target, and write a ready-to-post PR comment with --markdown <path>.

Why it matters: the 2026-07-28 MCP spec ships breaking changes (dropped initialize handshake, error-code and JSON-Schema shifts, HTTP+SSE → Streamable HTTP). A recorded .mcpvcr baseline turns "did we break our clients?" into a one-line CI check. See examples/contract-gate.yml for the full PR workflow with auto-commenting.

Use cases

MCP server development — Run mcp-probe in your test suite to catch regressions
CI/CD gates — Block deploys if your MCP server doesn't pass health checks
Server evaluation — Quickly assess third-party MCP servers before integrating them
Schema quality — Find missing descriptions and malformed schemas before users hit them

CI integration

mcp-probe exits 0 on full pass and 1 on any failure, so it drops directly into any CI pipeline:

yaml

# .github/workflows/mcp-health.yml
- name: Health-check MCP server
  run: npx @incultnitollc/mcp-probe test "$MCP_SERVER_CMD"

Use --json for structured output and jq to gate on specific metrics (e.g. fail the build if schemaWarnings > 0).

GitHub Action

Drop mcp-probe into your MCP server's GitHub Actions workflow in two lines:

yaml

- uses: incultnitollc/mcp-probe@v1
  with:
    command: 'node dist/index.js'

Gate your PRs on a publishability composite:

yaml

- uses: incultnitollc/mcp-probe@v1
  with:
    command: 'node dist/index.js'
    publishability: 'true'
    package: './package.json'
    fail-under: '70'

Inputs

Name	Required	Default	Description
`command`	yes	—	Command that launches your MCP server (e.g. `node dist/index.js` or `npx -y @your-scope/your-server`).
`fail-under`	no	`0`	Fail the job if the publishability composite drops below this value (0–100). Requires `publishability: 'true'`.
`publishability`	no	`false`	Run the publishability suite — 5 checks + 0–100 composite. Requires `mcp-probe >= 1.1.0` (ships 2026-05-23).
`package`	no	`''`	Path to `package.json` for the distribution-metadata check. Empty skips the distribution check.
`html-report`	no	`''`	Path to write the HTML scorecard. Upload via `actions/upload-artifact` in a follow-on step.
`mcp-probe-version`	no	`latest`	npm version, dist-tag, or `latest`. Pin for reproducible builds.
`json-output`	no	`''`	Path to write the JSON report for downstream parsing.

Outputs

Name	Description
`composite-score`	Publishability composite (0–100). Only set when `publishability: 'true'`.
`band`	Grade band: `publishable` / `almost` / `rough` / `not-ready`. Only set when `publishability: 'true'`.
`tools-pass-rate`	`tools_callable / tools_listed` as a decimal (e.g. `0.83`).
`schema-warnings`	Total schema warning count across all tools.

More examples: examples/basic.yml · examples/publishability-gate.yml · examples/matrix.yml.

Marketplace listing: github.com/marketplace/actions/mcp-probe-mcp-server-health-check.

Compared to MCP Inspector

The official MCP Inspector is a GUI for interactive exploration — point, click, see what a server returns. mcp-probe is a CLI for automated, repeatable diagnosis — every tool/resource/prompt called automatically, pass/fail scorecard out, exit code in. Use Inspector when you're exploring; use mcp-probe in CI, in pre-publish checks, or when you want a shareable scorecard of someone else's server.

Ecosystem

MCP Registry — Cross-source catalog of MCP servers (~6,900 indexed across 6 upstream lists) with quality scores powered by mcp-probe. CLI: npm i -g @incultnitollc/mcpr. Built by Incultnito LLC.

Development

bash

git clone https://github.com/incultnitollc/mcp-probe.git
cd mcp-probe
npm install
npm run dev -- test "npx -y @modelcontextprotocol/server-everything"
npm test

License

MIT - Incultnito LLC

Install

Add to VS Code Add to Cursor

claude_desktop_config.json

json

{
  "mcpServers": {
    "mcp-probe": {
      "command": "npx",
      "args": [
        "-y",
        "@incultnitollc/mcp-probe@1.2.0",
        "serve"
      ]
    }
  }
}

🛠️ Key Features

🚀 Use Cases

⚡ Developer Benefits

⚠️ Limitations

Topics

mcp-probe

Test your MCP server in 30 seconds

Install

Usage

Local stdio server

Remote server (Streamable HTTP)

Remote server (SSE)

Authenticated remote server

Options

Exit codes

JSON output

How tool calling works

Publishability score (v1.1.0+)

What scores look like on real servers

CI gate

Pre-publish vs install-time

Contract testing — "VCR for MCP" (v1.2.0+)

Use cases

CI integration

GitHub Action

Inputs

Outputs

Compared to MCP Inspector

Ecosystem

Development

License

Install

Related servers