@gaffer-sh/mcp
MCP (Model Context Protocol) server for Gaffer - give your AI assistant memory of your tests. Full setup guide: gaffer.sh/docs/mcp.
What is this?
This MCP server connects AI coding assistants like Claude Code and Cursor to your Gaffer test history and coverage data. It runs in code mode: three MCP tools over a namespace of 17 functions — 16 read-only analytics functions plus upload_test_results. It allows AI to:
- Check your project's test health (pass rate, flaky tests, trends)
- Look up the history of specific tests to understand stability
- Get context about test failures when debugging
- Analyze code coverage and identify untested areas
- Browse your projects (with a user API Key or sign-in)
- Access test report files (HTML reports, coverage, etc.)
Prerequisites
- A Gaffer account with test results uploaded
- For Claude Code, claude.ai and Claude Desktop on the hosted endpoint, nothing else: you sign in. For Cursor, CI and the npm package, an API Key from Account Settings > API Keys (app.gaffer.sh/account/api-keys)
Setup
There are two ways to connect. Use the hosted endpoint unless your client only supports local stdio servers.
Most people need no npm package: Claude Code, claude.ai and Claude Desktop connect to the hosted endpoint and sign in.
| Hosted endpoint | npm package |
|---|
| Install | Nothing | npx runs @gaffer-sh/mcp on your machine |
| Credential | OAuth sign-in, or Authorization: Bearer <key> header | GAFFER_API_KEY environment variable |
execute_code runs | In an isolated sandbox with no network access | In a local Node process |
| Functions | 17 with a key, same as npm. A signed-in connection has 16: it is read-only, so upload_test_results is listed but refused | All 17 |
| Clients | Those that support remote HTTP servers | Any client that can start a stdio server |
Option 1: Hosted endpoint
The endpoint is https://app.gaffer.sh/mcp (Streamable HTTP, POST only). Each credential is limited to 1,000 requests per hour. Requests without a valid credential get a 401, and requests over the limit get a 429 with a Retry-After header.
Sign in, no key
Claude Code, claude.ai and Claude Desktop sign in with OAuth. You do not create an API key.
Claude Code
claude mcp add --transport http gaffer https://app.gaffer.sh/mcp
Then run /mcp inside Claude Code, select gaffer, and finish the sign-in in your browser.
claude.ai and Claude Desktop
- Open Customize > Connectors, click + Add, then Add custom connector. On Team and Enterprise plans an owner adds it under Organization settings > Connectors, and each member then clicks Connect.
- Enter a name and
https://app.gaffer.sh/mcp as the URL, then click Add and sign in to Gaffer.
- On the consent page, pick the organization to connect and approve.
Anthropic's steps for custom connectors are the reference if the menu names change.
Share it with your team
Commit this .mcp.json at the root of your repository. It holds no secret. Each teammate signs in once with /mcp and Claude Code asks them to approve the project server first.
{
"mcpServers": {
"gaffer": {
"type": "http",
"url": "https://app.gaffer.sh/mcp"
}
}
}
What signing in grants
- Access to one organization, the one you pick on the consent page. To use another organization, connect again and pick it.
- Read access only. A signed-in assistant can read test runs, failures, flaky tests and coverage. It cannot call
upload_test_results.
- The same limit of 1,000 requests per hour, counted per connected app.
To revoke access, open Account Settings > AI assistants at app.gaffer.sh/account/ai-assistants and click Disconnect next to the app under Connected apps.
With an API key
Use a key for CI, Cursor, or if you prefer one. A key can also upload results.
Claude Code:
claude mcp add --transport http gaffer https://app.gaffer.sh/mcp --header "Authorization: Bearer gaf_YOUR_KEY"
Cursor: add to .cursor/mcp.json in your project, or ~/.cursor/mcp.json for every project:
{
"mcpServers": {
"gaffer": {
"url": "https://app.gaffer.sh/mcp",
"headers": {
"Authorization": "Bearer gaf_YOUR_KEY"
}
}
}
}
Option 2: npm package
The package runs locally over stdio and reads your key from GAFFER_API_KEY.
Claude Code (CLI)
claude mcp add gaffer -e GAFFER_API_KEY=gaf_YOUR_KEY -- npx -y @gaffer-sh/mcp
Claude Code (Manual)
Add to your Claude Code settings (~/.claude.json or project .claude/settings.json):
{
"mcpServers": {
"gaffer": {
"command": "npx",
"args": ["-y", "@gaffer-sh/mcp"],
"env": {
"GAFFER_API_KEY": "gaf_YOUR_KEY"
}
}
}
}
Cursor
Add to .cursor/mcp.json in your project:
{
"mcpServers": {
"gaffer": {
"command": "npx",
"args": ["-y", "@gaffer-sh/mcp"],
"env": {
"GAFFER_API_KEY": "gaf_YOUR_KEY"
}
}
}
}
Claude Desktop
Open Settings > Developer > Edit Config, or edit the file directly:
- macOS:
~/Library/Application Support/Claude/claude_desktop_config.json
- Windows:
%APPDATA%\Claude\claude_desktop_config.json
- Linux:
~/.config/Claude/claude_desktop_config.json
{
"mcpServers": {
"gaffer": {
"command": "npx",
"args": ["-y", "@gaffer-sh/mcp"],
"env": {
"GAFFER_API_KEY": "gaf_YOUR_KEY"
}
}
}
}
Quit and restart Claude Desktop after saving. Node.js must be installed.
How this server works
This server uses code mode. Instead of exposing one MCP tool per API call, it exposes three tools plus a codemode namespace you call from JavaScript. Fewer tool definitions occupy the context window, and a single execution can chain several calls.
| MCP tool | What it does |
|---|
execute_code | Run JavaScript against codemode.<function>(). Max 20 API calls, 30s timeout. |
search_tools | Find available functions by keyword. An empty query lists all of them. |
list_projects | List projects. Registered for a user API Key (gaf_) and for sign-in, not for a project token (gfr_). |
const health = await codemode.get_project_health({ projectId: "proj_abc" });
if (health.flakyTestCount > 0) {
const flaky = await codemode.get_flaky_tests({ projectId: "proj_abc" });
return { health, flaky };
}
return { health };
Functions available via execute_code
| Function | Category | Description |
|---|
get_project_health | health | Health score, pass rate, flaky count, trend |
get_test_history | testing | Pass/fail history for a specific test |
get_flaky_tests | testing | Tests with high flip rates (pass↔fail) |
list_test_runs | testing | Recent test runs, filterable by commit/branch/status |
get_test_run_details | testing | Parsed individual results for one run |
get_failure_clusters | testing | Failed tests grouped by root cause |
get_slowest_tests | testing | Slowest tests by P95 duration |
compare_test_metrics | testing | Compare test performance between commits or runs |
search_failures | testing | Search failures by error or test-name pattern, or list all recent failures |
get_coverage_summary | coverage | Overall coverage metrics and trend |
get_coverage_for_file | coverage | Coverage for specific files or paths |
get_untested_files | coverage | Files below a coverage threshold |
find_uncovered_failure_areas | coverage | Files with low coverage AND test failures |
get_report | reports | Report file URLs for a test run |
get_report_browser_url | reports | Signed browser-navigable report URL (30 min) |
get_upload_status | uploads | Whether CI results are uploaded and processed |
upload_test_results | uploads | Upload test results (write) — rate-limited and audit-logged |
Every function except upload_test_results is read-only.
Function Reference
Every days parameter defaults to 30 and accepts up to 365. The server then reduces it to your plan's analytics window: 30 days on Free, 90 on Pro and 180 on Team. A larger value is not an error, and you get the window your plan allows.
list_projects
List all projects you have access to. A user API Key lists projects in every organization you belong to. Sign-in lists the projects in the one organization you connected. Not available with a project token.
- Input:
organizationId (optional), limit (optional, default: 50, max: 100)
- Returns: List of projects with IDs, names, and organization info
- Example: "What projects do I have in Gaffer?"
get_project_health
Get the health metrics for a project.
- Input:
projectId (required for gaf_ keys), days (optional, default: 30, max: 365)
- Returns: Health score (0-100), pass rate, test run count, flaky test count, trend
- Example: "What's the health of my test suite?"
get_test_history
Get the pass/fail history for a specific test.
- Input:
projectId (required for gaf_ keys), testName, filePath or fullName (at least one), limit (optional, default: 20, max: 100; at most 50 on the Free plan). fullName is the exact value from get_flaky_tests and keeps Playwright projects apart
- Returns: History of runs with status, duration, branch, commit, and the failure
message for failed runs
- Example: "Is the login test flaky? Check its history"
get_flaky_tests
Get the list of flaky tests in a project.
- Input:
projectId (required for gaf_ keys), threshold (optional, default: 0.1), days (optional, default: 30, max: 365), limit (optional, default: 50, max: 100)
- Returns: List of flaky tests with
name, fullName (tells apart entries that share a name, one per Playwright project), flip rates, transition counts, run counts, flakinessScore and lastSeen
- Example: "Which tests are flaky in my project?"
list_test_runs
List recent test runs with optional filtering.
- Input:
projectId (required for gaf_ keys), commitSha (optional), branch (optional), status (optional), limit (optional, default: 20, max: 100), offset (optional, default: 0)
- Returns: List of test runs with pass/fail/skip counts, commit and branch info, plus
pagination (total, hasMore)
- Example: "What tests failed in the last commit?"
get_test_run_details
Get parsed test results for a specific test run.
- Input:
testRunId (required), projectId (required for gaf_ keys), status (optional filter), limit (optional, default: 100, max: 500), offset (optional, default: 0)
- Returns: Individual test results with name, status, duration, file path,
suite, line, retryAttempt, error and full errorStack; a summary with passed/failed/skipped/flaky/total counts
- Example: "Show me all failed tests from this test run"
get_report
Get URLs for report files uploaded with a test run.
- Input:
testRunId (required). Not available with a project token (gfr_)
- Returns: List of files with filename, size, content type, download URL
- Example: "Get the Playwright report for the latest test run"
get_report_browser_url
Get a browser-navigable URL for viewing a test report.
- Input:
projectId (required for gaf_ keys), testRunId (required), filename (optional)
- Returns: Signed URL valid for 30 minutes
- Example: "Give me a link to view the test report"
get_slowest_tests
Get the slowest tests in a project, sorted by P95 duration.
- Input:
projectId (required for gaf_ keys), days (optional, default: 30, max: 365), limit (optional, default: 20, max: 100), framework (optional), branch (optional)
- Returns: List of tests with average and P95 duration, run count
- Example: "Which tests are slowing down my CI pipeline?"
compare_test_metrics
Compare test metrics between two commits or test runs.
- Input:
projectId (required for gaf_ keys), testName (required), beforeCommit/afterCommit OR beforeRunId/afterRunId
- Returns: Before/after metrics with duration change, percentage and whether the status changed
- Example: "Did my fix make this test faster?"
get_coverage_summary
Get the coverage metrics summary for a project.
- Input:
projectId (required for gaf_ keys), days (optional, default: 30, max: 365)
- Returns: Line/branch/function coverage percentages, trend, report count, lowest coverage files
- Example: "What's our test coverage?"
get_coverage_for_file
Get coverage metrics for specific files or paths.
- Input:
projectId (required for gaf_ keys), filePath (required - exact or partial match)
- Returns: Up to 10 matching files with line/branch/function coverage
- Example: "What's the coverage for our API routes?"
get_untested_files
Get files with little or no test coverage.
- Input:
projectId (required for gaf_ keys), maxCoverage (optional, default: 10%), limit (optional, default: 20, max: 100)
- Returns: List of files below threshold sorted by coverage (lowest first)
- Example: "Which files have no tests?"
find_uncovered_failure_areas
Find code areas with both low coverage AND test failures (high risk).
- Input:
projectId (required for gaf_ keys), days (optional, default: 30, max: 365), coverageThreshold (optional, default: 80%)
- Returns: Risk areas ranked by score, with file path, coverage %, failure count
- Example: "Where should we focus our testing efforts?"
get_failure_clusters
Group failed tests by root cause using error message similarity.
- Input:
projectId (required for gaf_ keys), testRunId (required)
- Returns: Clusters of failed tests grouped by similar error messages, with representative error and test count
- Example: "Are these 15 failures from the same bug?"
search_failures
Search past failures by error message, stack trace, or test name — or list every failure in the window.
- Input:
query (optional, omit to return all failures), projectId (required for gaf_ keys), searchIn (optional: errors/names/all, default all), days (optional, default: 30, max: 365), branch (optional), limit (optional, default: 20, max: 100), offset (optional, default: 0)
- Returns: Matching failures with test name, error message, full
errorStack, run and commit context, plus pagination (limit, offset, hasMore) and truncated when scan caps cut the list short
- Example: "Have we seen this connection-refused error before?" / "What failed in the last 7 days?"
get_upload_status
Check if CI results have been uploaded and processed.
- Input:
projectId (required for gaf_ keys), sessionId (optional), commitSha (optional), branch (optional)
- Returns: Upload session(s) with processing status, linked test runs and coverage reports
- Example: "Are my test results ready for commit abc123?"
upload_test_results
Upload structured test results. This is the only function that writes.
Use it when you have results in hand — parsed from CI output or a runner's JSON
report — and no Gaffer CLI is available to upload them.
- Input:
projectId (required for gaf_ keys), framework (required), tests (required), branch, commitSha, ciProvider, startedAt, finishedAt, coverage
- Returns:
uploadSessionId, the generated runId, and the derived pass/fail/skip summary
- Example: "Upload these 42 parsed pytest results so we can track them"
runId, the run timestamps and the summary are derived from tests — pass
startedAt/finishedAt only if you know the real wall-clock window.
Two constraints worth knowing:
- Not idempotent. Each call creates a new run, so a retry after an uncertain
failure produces a duplicate. Check
get_upload_status instead of retrying.
- Rate-limited per project, and every call is written to the project's audit
log with the id of the credential that made it.
Processing is asynchronous: results take a few seconds to become visible to the
read functions.
Agentic CI Workflows
These workflows show how an AI agent diagnoses CI failures, waits for results, and finds coverage gaps. Each step is a codemode function, so a whole chain runs inside one execute_code call rather than one round-trip per step.
Workflow: Diagnose CI Failures
list_test_runs(projectId, status="failed")
→ get_test_run_details(projectId, testRunId, status="failed")
→ get_failure_clusters(projectId, testRunId)
→ get_test_history(projectId, testName="...")
→ compare_test_metrics(projectId, testName, beforeCommit, afterCommit)
- Find the failed test run
- Get individual failure details with stack traces
- Group failures by root cause — often 15 failures are 2-3 bugs
- Check if each failure is new (regression) or recurring
- Verify fixes by comparing before/after
Workflow: Wait for Results
get_upload_status(projectId, commitSha="abc123")
→ poll until processingStatus="completed"
→ get_test_run_details(projectId, testRunId)
- Check if results for a commit have been uploaded
- Wait for processing to complete
- Use linked test run IDs to get results
Workflow: Find Coverage Gaps
find_uncovered_failure_areas(projectId)
→ get_untested_files(projectId)
→ get_coverage_for_file(projectId, filePath="src/critical/")
- Find files with both low coverage and test failures (highest risk)
- Find files with no coverage at all
- Drill into specific directories for targeted analysis
Function Quick Reference
| Agent Question | Function |
|---|
| "What failed?" | get_test_run_details |
| "Same root cause?" | get_failure_clusters |
| "Seen this error before?" | search_failures |
| "Is it flaky?" | get_flaky_tests |
| "Is this new?" | get_test_history |
| "Did my fix work?" | compare_test_metrics |
| "Are results ready?" | get_upload_status |
| "What's untested?" | find_uncovered_failure_areas |
| "What's slow?" | get_slowest_tests |
Prioritizing Coverage Improvements
When using coverage tools to improve your test suite, combine coverage data with codebase exploration for best results:
1. Understand Code Utilization
Before targeting files purely by coverage percentage, explore which code is actually critical:
- Find entry points: Look for route definitions, event handlers, exported functions - these reveal what code actually executes in production
- Find heavily-imported files: Files imported by many others are high-value targets
- Identify critical business logic: Look for files handling auth, payments, data mutations, or core domain logic
2. Prioritize by Impact
Low coverage alone doesn't indicate priority. Consider:
- High utilization + low coverage = highest priority - Code that runs frequently but lacks tests
- Large files with 0% coverage - More uncovered lines means bigger impact on overall coverage
- Files with both failures and low coverage - Use
find_uncovered_failure_areas for this
3. Use Path-Based Queries
The get_untested_files tool may return many frontend components. For backend or specific areas:
# Query specific paths with get_coverage_for_file
get_coverage_for_file(filePath="server/services")
get_coverage_for_file(filePath="src/api")
get_coverage_for_file(filePath="lib/core")
4. Iterative Improvement
- Get baseline with
get_coverage_summary
- Identify targets with
get_coverage_for_file on critical paths
- Write tests for highest-impact files
- Re-check coverage after CI uploads new results
- Repeat
Authentication
There are three credentials.
| Credential | Prefix | Scope | What it can do |
|---|
| OAuth sign-in | none | One organization, chosen at sign-in | Read functions only. No upload_test_results. list_projects lists that organization's projects. Hosted endpoint only. |
| User API key | gaf_ | Every project your account can reach | Read functions, plus upload_test_results. list_projects lists projects in every organization you belong to. |
| Project token | gfr_ | One project | Read functions and upload_test_results for that project. Omit projectId. |
OAuth sign-in
On the hosted endpoint, Claude Code, claude.ai and Claude Desktop sign in without a key. The grant covers one organization and is read-only: it cannot call upload_test_results. Revoke it under Account Settings > AI assistants > Connected apps.
User API Keys
User API Keys (gaf_ prefix) cover every project your account can reach. They can read all of them and upload test results to any of them (upload_test_results, rate-limited to 300 per hour per project). Get your API Key from: Account Settings > API Keys
Project Tokens
Project Tokens (gfr_ prefix) are designed for uploading test results and only provide access to a single project. When you use one, omit projectId: it resolves automatically. list_projects and get_report are not available with one. A user API Key or sign-in is preferred for the MCP server because both enable list_projects.
Environment Variables
| Variable | Required | Description |
|---|
GAFFER_API_KEY | Yes | Your Gaffer API Key (starts with gaf_) |
GAFFER_API_URL | No | API base URL (default: https://app.gaffer.sh) |
Local Development
Test locally with Claude Code (use absolute path to built file):
{
"mcpServers": {
"gaffer": {
"command": "node",
"args": ["/absolute/path/to/dist/index.js"],
"env": {
"GAFFER_API_KEY": "gaf_..."
}
}
}
}
License
MIT