io.github.AishwaryShrivastav/vibe-testing

Official

by AishwaryShrivastav · TypeScript

Vibe Test — Browser Testing Agent

Code-aware browser testing agent — 13 Playwright tools for AI code editors via MCP

Browser testing agent with 13 Playwright tools for AI-powered code editing and automation.

Topics

ai-testingbrowser-testingclaude-codecursore2e-testingmcpmcp-servermodel-context-protocolnpxplaywrightqatest-automationtestingtypescriptvibe-codingai-agentai-toolscursor-idewindsurf

README.md21 KBView on GitHub

vibe-test

Code-aware browser testing agent for AI-powered editors.

Reads your codebase, understands every route and form, opens a real Playwright browser, explores every element, and reports what works and what breaks — with screenshots.

Works as an MCP server that gives your AI editor (Claude Code, Cursor, Windsurf, VS Code Copilot) 13 browser testing tools — or as a standalone CLI.

Install in Any Project (One Command)

bash

cd /path/to/your/project
npx vibe-testing@latest init

This command:

Detects which AI editors you have installed
Registers vibe-test in global editor configs (~/.claude/settings.json, ~/.cursor/mcp.json, etc.) so the tools are available in every project, every session
Creates project-level MCP configs and AI instruction files
Auto-detects your app's URL (reads .env, vite.config, framework defaults)
Creates VIBE.md (edit with your test credentials) and vibe.config.json

Then open your editor and say:

"Scan this codebase and test it against http://localhost:3000"

Your AI will pick up the tools automatically and start testing.

How It Works
MCP Setup (per editor)
MCP Tools Reference
Recommended Workflow
init Command
CLI Commands
VIBE.md — Project Guidance
Configuration (vibe.config.json)
Supported Frameworks
FAQ

How It Works

code

npx vibe-testing@latest init
       ↓
Registers 13 MCP tools in your editor
       ↓
You ask: "Test the checkout flow"
       ↓
AI calls: scan_codebase → get_context("checkout") → login → explore_page → execute_scenario → generate_report
       ↓
HTML report opens in browser with screenshots of every step

No test cases to write. The AI reads your source code to understand real field names and routes, opens a browser, tests everything, and shows you what's broken.

MCP Setup

Option 1 — Automatic (recommended)

bash

npx vibe-testing@latest init

Detects and configures all installed editors. Done.

Option 2 — Manual per editor

Claude Code

Add to ~/.claude/settings.json (global — works in every project):

json

{
  "mcpServers": {
    "vibe-test": {
      "command": "npx",
      "args": ["-y", "vibe-testing@latest", "--mcp"]
    }
  }
}

Or add to .mcp.json in your project root (project-level only):

json

{
  "mcpServers": {
    "vibe-test": {
      "command": "npx",
      "args": ["-y", "vibe-testing@latest", "--mcp"]
    }
  }
}

Cursor

Add to ~/.cursor/mcp.json (global) or .cursor/mcp.json (project):

json

{
  "mcpServers": {
    "vibe-test": {
      "command": "npx",
      "args": ["-y", "vibe-testing@latest", "--mcp"]
    }
  }
}

Windsurf

Add to ~/.codeium/windsurf/mcp_config.json:

json

{
  "mcpServers": {
    "vibe-test": {
      "command": "npx",
      "args": ["-y", "vibe-testing@latest", "--mcp"]
    }
  }
}

VS Code (GitHub Copilot)

Add to .vscode/mcp.json in your project:

json

{
  "servers": {
    "vibe-test": {
      "command": "npx",
      "args": ["-y", "vibe-testing@latest", "--mcp"]
    }
  }
}

Roo Code / Cline

Add to .roo/mcp.json:

json

{
  "mcpServers": {
    "vibe-test": {
      "command": "npx",
      "args": ["-y", "vibe-testing@latest", "--mcp"]
    }
  }
}

From local build (development)

json

{
  "mcpServers": {
    "vibe-test": {
      "command": "node",
      "args": ["/path/to/vibe-testing/dist/mcp-server.js"]
    }
  }
}

MCP Tools Reference

13 tools available to your AI editor after setup:

Tool	When to call	Returns
`scan_codebase`	Always first. Reads source code, finds routes/forms/tests/gaps	Routes, forms, coverage map, generated scenarios, `route_changes` since last scan
`get_context`	Before writing test steps. Returns source files for a feature	Actual source code with real field names and selectors
`login`	When app requires authentication	Post-login screenshot, token state, API calls observed
`scan_page_elements`	To see all interactive elements on a page	Element list with selectors + page screenshot
`explore_page`	Broad "does everything work?" testing	Interaction results, API calls, errors, screenshot
`execute_scenario`	Run specific test steps	Step-by-step logs + screenshots
`get_coverage`	View coverage map and untested routes	Coverage entries, gaps, available scenarios
`suggest_tests`	Find coverage gaps after exploration	Prioritized, ready-to-run scenarios with steps
`take_screenshot`	Quick visual verification	Screenshot of any URL
`generate_report`	Build HTML report (auto-opens)	Report path + summary
`run_full_test`	One-shot: scan → execute → explore → report	Full results + `snapshot_diff` vs last run
`run_converge`	Iterative testing until thresholds	Summary across all rounds + `snapshot_diff` vs last run
`cleanup`	Close browsers, free resources	—

Tool Inputs

scan_codebase

json

{
  "codebase_path": "/path/to/project",
  "url": "http://localhost:3000",
  "mode": "deep"
}

get_context

json

{ "feature": "login" }
{ "feature": "/checkout" }
{ "feature": "user profile form" }

login

json

{
  "email": "test@example.com",
  "password": "TestPass123!",
  "login_url": "/login"
}

scan_page_elements / explore_page

json

{
  "route": "/dashboard",
  "authenticated": true
}

execute_scenario

json

{
  "scenario": {
    "id": "create-item",
    "name": "Create a new item",
    "route": "/items",
    "steps": [
      { "action": "navigate", "url": "/items", "description": "Open items page" },
      { "action": "click", "selector": "text=Add Item", "description": "Open create form" },
      { "action": "fill", "selector": "[name='title']", "value": "Test Item", "description": "Fill title" },
      { "action": "fill", "selector": "[name='description']", "value": "Test description", "description": "Fill description" },
      { "action": "click", "selector": "button[type='submit']", "description": "Submit form" }
    ],
    "expected_outcome": "New item appears in the list",
    "requires_auth": true
  }
}

Step actions: navigate, fill, click, select, wait, assert, upload

take_screenshot

json

{ "url": "/settings", "authenticated": true, "full_page": false }

run_full_test

json

{ "url": "http://localhost:3000", "codebase_path": "/path/to/project", "mode": "deep" }

run_converge

json

{
  "url": "http://localhost:3000",
  "max_followup_rounds": 4,
  "target_pass_rate": 0.92,
  "max_high_severity_gaps": 2
}

Recommended Workflow

Full test session

Tell your AI editor:

code

Scan this codebase and test it against http://localhost:3000.
Log in with test@example.com / pass123. Explore the dashboard and
settings pages, run the suggested tests, and generate a report.

The AI will:

scan_codebase — understand routes, forms, existing tests
get_context("login") — read actual login form source code
login — authenticate in a real browser
explore_page("/dashboard") — click everything, observe what breaks
explore_page("/settings") — same
suggest_tests — find coverage gaps
execute_scenario × N — run targeted test flows
generate_report — HTML report opens automatically
cleanup — close browsers

Test a specific feature

code

Test the checkout flow using vibe-test. Get context for checkout,
then run the full purchase flow with card number 4242424242424242.

The AI will:

scan_codebase (if not already done)
get_context("checkout") — read CheckoutForm.tsx, api/orders/route.ts etc.
login — authenticate
execute_scenario — fill the real form fields from source code
generate_report

Verify a bug fix

code

I fixed the login redirect bug. Use vibe-test to confirm it's working.

The AI will:

login — test the login flow
take_screenshot — visual confirmation of the post-login state
Report back what it sees

Find what's broken

code

Explore every page and tell me what's broken.

The AI will run explore_page on every route, collecting API errors, broken elements, and failed interactions, then suggest_tests with the broken items marked as high priority.

init Command

bash

npx vibe-testing@latest init [options]

What it creates:

File	Where	Purpose
`.mcp.json`	Project root	Claude Code MCP config (project-level)
`~/.claude/settings.json`	Global	Claude Code MCP config (all projects)
`.cursor/mcp.json`	Project root	Cursor MCP config
`~/.cursor/mcp.json`	Global	Cursor MCP config (all projects)
`.cursor/rules/vibe-test.mdc`	Project	Cursor rules — `alwaysApply: true`
`.windsurfrules`	Project	Windsurf instructions
`~/.codeium/windsurf/mcp_config.json`	Global	Windsurf MCP config (all projects)
`.vscode/mcp.json`	Project	VS Code Copilot MCP config
`.github/copilot-instructions.md`	Project	GitHub Copilot instructions
`.roo/mcp.json`	Project	Roo Code MCP config
`CLAUDE.md`	Project	Claude Code session instructions
`AGENTS.md`	Project	Universal agent instructions (Codex, Devin, Zed)
`VIBE.md`	Project	Test guidance — edit with your credentials
`vibe.config.json`	Project	Config — URL auto-detected from your project

Options:

bash

npx vibe-testing@latest init                     # auto-detect editors, register globally + project
npx vibe-testing@latest init --no-global         # project-level only, skip global registration
npx vibe-testing@latest init --editor cursor     # only configure Cursor
npx vibe-testing@latest init --editor claude-code windsurf

After init, edit VIBE.md with your login URL and test credentials.

CLI Commands

bash

# Set up in current project
npx vibe-testing@latest init

# Run tests against a URL
npx vibe-testing@latest run http://localhost:3000
npx vibe-testing@latest run https://staging.myapp.com --mode deep
npx vibe-testing@latest run http://localhost:3000 --codebase /path/to/project --scope /login /dashboard

# Iterative testing until coverage thresholds
npx vibe-testing@latest converge http://localhost:3000
npx vibe-testing@latest converge http://localhost:3000 --max-rounds 6 --target-pass-rate 0.95

# Open last report in browser
npx vibe-testing@latest report

# Reset memory and screenshots for a clean run
npx vibe-testing@latest reset

`run` options

Option	Default	Description
`--mode fast\|deep`	`deep`	`fast`: quick scan. `deep`: full feature extraction + exploration
`--no-headed`	—	Run browser headless (default: visible)
`--codebase <path>`	cwd	Path to project root
`--scope <routes...>`	all	Test only specific routes
`-c <path>`	`vibe.config.json`	Config file path

`converge` options

Option	Default	Description
`--max-rounds <n>`	`4`	Max follow-up rounds after baseline
`--target-pass-rate <r>`	`0.92`	Stop when pass rate ≥ this (0–1)
`--max-gaps <n>`	`2`	Stop when critical+important gaps ≤ this

VIBE.md — Project Guidance

Create VIBE.md in your project root. Vibe Test reads it automatically on every run.

markdown

## Login URL
/login

## Test Credentials
- Email: test@example.com
- Password: TestPass123!

## Never Automate
- delete account
- cancel subscription
- [data-testid="danger-zone"]
- .billing-section

## Known Flaky
- /notifications (WebSocket dependent — skip or expect retry)
- /live-feed

## Notes
- Admin panel at /admin — use admin@example.com / adminpass
- Dashboard data loads async — wait for [data-loaded="true"]
- Profile page: click "Edit Profile" before form fields appear

See VIBE.example.md for the full template.

Configuration

vibe.config.json

Created automatically by init with auto-detected URL. Edit as needed:

json

{
  "url": "http://localhost:3000",
  "mode": "deep",
  "auth": {
    "strategy": "credentials",
    "login_url": "/login",
    "credentials": {
      "email": "test@example.com",
      "password": "TestPass123!"
    }
  },
  "never_interact": [
    "delete account",
    "cancel subscription",
    "[data-testid='danger-zone']"
  ],
  "scope": {
    "include": ["/**"],
    "exclude": ["/admin/**", "/api/**"],
    "max_routes": 30,
    "seed_routes": ["/live/dev-mode-a-now"]
  },
  "browser": {
    "headed": true,
    "slowMo": 40,
    "timeout": 30000
  }
}

Key	Description
`url`	App URL — localhost or staging. Auto-detected by `init`.
`mode`	`fast` (heuristic scan) or `deep` (full extraction + exploration)
`auth.strategy`	`credentials` (form login), `basic` (HTTP Basic Auth), or `skip`
`auth.credentials`	Login credentials — persisted across runs once used
`never_interact`	Text patterns or CSS selectors to skip during exploration
`scope.exclude`	Route patterns to exclude from testing
`scope.max_routes`	Cap how many routes are tested per run
`scope.seed_routes`	Concrete URLs for dynamic-segment routes the parser can't enumerate (e.g. `/live/[slug]` → `/live/dev-mode-a-now`). Each seeded route inherits `requires_auth` and the source file from its dynamic parent.
`browser.headed`	`true` = visible browser. CLI default `true`, MCP server default `false` (headless) so editor sessions aren't disrupted by pop-up windows.
`browser.slowMo`	Milliseconds between actions (useful for debugging)
`routes`	`auto` (default) discovers routes from the codebase. `config` uses only routes explicitly listed in config.

Supported Frameworks

Framework	Routes	API endpoints	Forms
Next.js App Router	✅	✅	✅
Next.js Pages Router	✅	✅	✅
Next.js (src/ variant)	✅	✅	✅
React SPA (react-router)	✅	—	✅
Vue + Vite (vue-router)	✅	—	✅
Nuxt	✅	✅	✅
SvelteKit	✅	✅	✅
Express / Fastify	—	✅	✅
Monorepos (Turborepo, pnpm, Lerna)	✅	✅	✅

Existing test files are also read to build a coverage map:

Test runner	Supported
Jest / Vitest	✅
Playwright	✅
Cypress	✅

Self-Improvement

Vibe Test learns across runs and stores intelligence in .vibe/:

Working selectors — remembers [name='email'] worked on /login, uses it next run
Route timings — adjusts timeouts based on measured load times
Auth credentials — saved after first login, reused automatically
Flaky routes — tracks high fail-rate routes, marks them for retry
Skip routes — routes that consistently error (need URL params) are auto-skipped
Route manifest (.vibe/route-manifest.json) — every scan diffs against the previous one; new and removed routes are surfaced as route_changes on scan_codebase results so the AI can immediately cover them
Run snapshot (.vibe/run-snapshot.json) — every run captures per-route pass/fail status and diffs against the prior run; snapshot_diff flags newly_passing (fixes), newly_failing (regressions), still_failing, plus added/removed routes

Reset with npx vibe-testing@latest reset to start fresh.

Regression detection in action

After a second run, the console and VibeRunResult include a snapshot diff:

code

─── Changes since last run ───
  ✓ Fixed:      /login
  ✗ Regression: /checkout
  ℹ New:        /admin/users

json

{
  "snapshot_diff": {
    "newly_passing": ["/login"],
    "newly_failing": ["/checkout"],
    "still_failing": [],
    "new_routes": ["/admin/users"],
    "removed_routes": []
  }
}

run_converge returns the same shape, so iterative runs in your editor highlight what you just broke.

How the AI Uses These Tools

When you ask your editor to "test the login flow", here is exactly what it does:

code

User: "Test the login flow"

AI calls:
  scan_codebase({ codebase_path: ".", url: "http://localhost:3000" })
    → Finds /login route, LoginForm component, POST /api/auth/login endpoint
    → Returns 8 generated test scenarios

  get_context({ feature: "login" })
    → Returns src/app/login/page.tsx (has email, password fields, name="email", name="password")
    → Returns src/app/api/auth/login/route.ts (POST handler, returns { token })
    → AI now knows the REAL selectors: [name='email'], [name='password']

  login({ email: "test@example.com", password: "pass123" })
    → Opens Chromium, navigates to /login
    → Fills email and password fields
    → Clicks submit
    → Returns: { success: true, final_url: "/dashboard", tokens_found: 2 }
    → Returns screenshot of post-login dashboard

  execute_scenario({
    scenario: {
      name: "Login with invalid password",
      steps: [
        { action: "navigate", url: "/login" },
        { action: "fill", selector: "[name='email']", value: "test@example.com" },
        { action: "fill", selector: "[name='password']", value: "wrongpassword" },
        { action: "click", selector: "button[type='submit']" }
      ],
      expected_outcome: "Error message shown"
    }
  })
    → Returns screenshot showing error state

  generate_report()
    → Writes .vibe/report.html
    → Opens in browser automatically

AI reports: "Login works. Invalid password shows an error. All 3 login scenarios passed."

FAQ

Does vibe-test use an AI/LLM internally? No. It uses heuristic verification (URL changes, toast detection, API errors). Your editor's AI (Claude, GPT-4, etc.) is the brain — it sees screenshots and decides what to test next.

What's the difference between explore_page and execute_scenario? explore_page is broad — it clicks every button and input it finds and reports the results. execute_scenario is precise — you give it specific steps and it follows them exactly. Use explore_page to find what's on a page, then execute_scenario to test specific flows.

What's get_context for? It returns the actual source code for a feature — so the AI knows [name='email'] instead of guessing #email-input. Always call it before writing test steps for a specific feature.

Does it handle SPAs with client-side routing? Yes. Playwright navigates the real browser, so client-side routing (React Router, Vue Router, etc.) works naturally.

Does it handle login / authentication? Yes. The login tool fills credentials in a real browser, captures auth tokens from localStorage/cookies, and keeps that session alive for authenticated tests. Credentials are persisted in .vibe/memory/ and reused automatically.

Will it click "Delete Account" or other destructive buttons? No. Set never_interact in vibe.config.json or VIBE.md to blocklist dangerous actions. Any button whose text or selector matches is skipped during exploration.

Can I use it without an AI editor? Yes — vibe-test run https://your-app.com runs standalone. It scans, generates scenarios, executes them, and produces an HTML report without needing an editor.

How do I test a staging environment? Set url in vibe.config.json to your staging URL, or pass it as a CLI argument: npx vibe-testing@latest run https://staging.myapp.com.

Does it work with monorepos? Yes. init detects Turborepo/pnpm/yarn workspaces and finds the frontend app automatically.

Requirements

Node.js ≥ 20 (the test suite uses vitest 4.x which requires Node 20+)
Playwright Chromium — install once with:
bash
```
npx playwright install chromium
```
(vibe-test will prompt you if it's missing)

Docker

A Node 20 + Chromium image is included for environments that prefer container-based MCP servers (and for Glama.ai quality scoring):

bash

docker build -t vibe-test .
# wire into your editor's MCP config:
# { "command": "docker", "args": ["run", "--rm", "-i", "vibe-test"] }

Contributing

bash

git clone https://github.com/AishwaryShrivastav/vibe-testing.git
cd vibe-testing
npm install
npx playwright install chromium
npm run build   # tsc → dist/
npm run dev     # run CLI without building
npm run mcp     # run MCP server without building

See CHANGELOG.md for version history.

License

MIT — Aishwary Shrivastav

Install

claude_desktop_config.json

{
  "mcpServers": {
    "vibe-testing": {
      "command": "npx",
      "args": [
        "-y",
        "vibe-testing@0.4.2"
      ]
    }
  }
}

io.github.AishwaryShrivastav/vibe-testing

Topics

vibe-test

Install in Any Project (One Command)

Contents

How It Works

MCP Setup

Option 1 — Automatic (recommended)

Option 2 — Manual per editor

Claude Code

Cursor

Windsurf

VS Code (GitHub Copilot)

Roo Code / Cline

From local build (development)

MCP Tools Reference

Tool Inputs

Recommended Workflow

Full test session

Test a specific feature

Verify a bug fix

Find what's broken

init Command

CLI Commands

`run` options

`converge` options

VIBE.md — Project Guidance

Configuration

vibe.config.json

Supported Frameworks

Self-Improvement

Regression detection in action

How the AI Uses These Tools

FAQ

Requirements

Docker

Contributing

License

Links

Install

Topics

vibe-test

Install in Any Project (One Command)

Contents

How It Works

MCP Setup

Option 1 — Automatic (recommended)

Option 2 — Manual per editor

Claude Code

Cursor

Windsurf

VS Code (GitHub Copilot)

Roo Code / Cline

From local build (development)

MCP Tools Reference

Tool Inputs

Recommended Workflow

Full test session

Test a specific feature

Verify a bug fix

Find what's broken

init Command

CLI Commands

run options

converge options

VIBE.md — Project Guidance

Configuration

vibe.config.json

Supported Frameworks

Self-Improvement

Regression detection in action

How the AI Uses These Tools

FAQ

Requirements

Docker

Contributing

License

Links

Install

`run` options

`converge` options