io.github.thatsrajan/vidlens-mcp

A prompt asks VidLens to review five DJI Pocket 4P videos; the view pulls back to the full answer — verdict, spec table, ranked videos and an audience sentiment chart

Your AI can read the web.
Now it can also watch it.

npx vidlens-mcp setup

• Research video: search, transcribe, compare, and cite YouTube and public social video.
• Build a local library: import once, then search transcripts, comments, frames, and OCR without fetching the same video again.
• Inspect visual evidence: find the exact frame, timestamp, match reason, and source in a visual gallery plus structured client output.

VidLens is an MCP server that gives your AI agent eyes on video — YouTube, X, TikTok, Instagram, other video pages, or a file on your disk. Paste a link and ask a question. VidLens reads the transcript, looks at the frames, and answers with timestamps you can check. Everything it ingests lands in a library on your own machine, so your agent never has to watch the same video twice.

No API keys to start. Works in Claude Desktop, Claude Code, and Codex. Capable clients can turn VidLens's structured results into native charts, cards, tables, and visual reports in their own response surfaces.

Watch VidLens in action

Watch: Your AI Can Read the Web. Now It Can Watch It. — a 1 minute 46 second VidLens demo

▶ Watch the 1:46 demo on YouTube

Try it

Once VidLens is set up, paste any of these into your AI client:

"Search YouTube for M5 Max MacBook Pro reviews. What do reviewers agree on, and where do they disagree?"

VidLens searches, reads the transcripts across reviewers, and synthesizes consensus and disagreement with sources and timestamps.

A prompt is sent, the agent works, and a three-point summary appears with a link to the source video

"Transcribe this video and summarize it: https://x.com/username/status/123..."

A single public X, Instagram, or TikTok video URL usually needs no API key at all. VidLens fetches it, transcribes it, and keeps the transcript so you can ask follow-up questions later — in this session or any future one.

VidLens summarizes OpenAI's latest Instagram Reel with the reel title, post date, and verification link

"Find the frame in this video where they show the benchmark chart."

Visual search looks at what is on screen — slides, charts, whiteboards, product shots — and shows the actual frame image with its timestamp in the local browser gallery, not just a text guess.

Native client presentation

VidLens returns portable structured evidence: sources, timestamps, scores, OCR, descriptions, provenance, and limitations. Codex, Claude, and other capable clients can compose that evidence into their own native charts, comparison cards, tables, and visual reports.

A prompt asks VidLens to compare the latest three Diary of a CEO episodes; the agent works through them and renders an inline report with view charts, per-episode sentiment bars, and a source provenance footer

The presentation belongs to the client rather than an embedded VidLens UI. For extracted-frame inspection, searchVisualContent can also open the browser gallery.

How it works

Video sources go in on the left. VidLens ingests them through whichever route works — the YouTube API when a key is configured, yt-dlp, or direct page extraction — and stores the results in a local library: transcripts, frames, and embeddings. Your agents query that library from any MCP client.

Two honesty guarantees are built in:

• Fallbacks, not failures. Every YouTube data tool tries multiple routes before giving up, so a missing key or an exhausted quota degrades gracefully instead of breaking.
• Provenance on responses. Tool responses report which route served the data and whether anything was partial. No silent degradation.

Everything lives in one directory on your disk. No external database, no Docker. Back it up by copying it; delete it to start fresh.

Install

Setup wizard (recommended)

npx vidlens-mcp setup

The wizard detects Claude Desktop, Claude Code, and Codex, checks the local dependencies VidLens needs, and offers one simple choice:

• Free setup (recommended): no API keys. YouTube search, transcripts, metadata, and the local library work immediately.
• Enhanced setup: add optional APIs, speech-to-text, web search, or browser cookies one at a time. Press Enter to skip anything you do not need; saved values are kept and never shown.

Press Enter to choose Free, or run npx vidlens-mcp setup --enhanced whenever you want Enhanced capabilities. For Claude Code, setup also registers VidLens in the user MCP registry, installs the shared VidLens workflow skill, and checks the result with claude mcp list when possible.

Manual configuration

Claude Desktop — add to ~/Library/Application Support/Claude/claude_desktop_config.json:

{
  "mcpServers": {
    "vidlens-mcp": {
      "command": "npx",
      "args": ["-y", "vidlens-mcp", "serve"]
    }
  }
}

Claude Code — prefer the wizard:

npx vidlens-mcp setup --client claude_code
claude mcp list

If you configure it by hand instead, add the same mcpServers.vidlens-mcp entry to ~/.claude.json.

Codex — use the setup helper so the MCP server and bundled VidLens skills are registered together:

npx vidlens-mcp setup --client codex

Restart your client

Fully quit and reopen Claude Desktop (⌘Q, not just close the window) — MCP servers load on startup. For Claude Code, start a new session or run /mcp after setup.

From a local checkout

npm install in a checkout does not put the binary on your PATH. Use npm run setup from the checkout, or npm install -g . / npm link if you want the bare vidlens-mcp command while developing.

Your library

VidLens is durable memory for video work, not a per-session scratchpad. Everything you import persists on disk under VIDLENS_DATA_DIR: transcript collections, comment collections, downloaded media, and visual indexes. Even the active collection is remembered.

Recall it in one call. At the start of a session, recallWorkspace returns a compact digest of everything already stored, so an agent knows what it has before searching or importing again. Tool descriptions carry the same reminder into every client, so agents check first instead of re-fetching.

Reuse by URL or asset key. An imported social video's transcript can be pulled up again by its original URL or by its asset key — no re-download, no re-transcription.

One shared library across agents. The default data directory is the same for every client, so Claude Desktop, Claude Code, and Codex all share one library — import a video in one, search it from another.

⚠️ Do not put VIDLENS_DATA_DIR inside Dropbox, iCloud, Google Drive, or any file-sync folder. VidLens uses SQLite (with WAL), and live databases under a file syncer get corrupted or spawn conflict copies. Keep the data dir on a local disk — the default location already does.

Beyond YouTube

For X, Instagram, TikTok, and generic video pages, VidLens uses a capability ladder: it takes the cheapest route that works, and an API key is an uplift, not a prerequisite.

On a mirrored iPhone, a prompt asks the agent what an X video shared and what its comments said

Three caveats, stated plainly:

• Browser assistance is host-level. The MCP server does not control a browser itself; it relies on whatever browser tooling your client provides, which differs by host.
• Seeing a video is not the same as exporting it. A post that plays in a browser does not guarantee automatic media export.
• Concrete URLs only. VidLens imports specific post or video URLs, never profiles, explore pages, hashtags, or popularity pages.

For social and local video work beyond plain download, install ffmpeg (brew install ffmpeg on macOS) — frame extraction, visual indexing, and STT audio chunking need it.

The tools — 47 across 11 modules

Explore — YouTube discovery and research (1)

Core — video and channel intelligence (7)

Knowledge base — semantic transcript search (7)

Sentiment and analysis (4)

Creator intelligence (4)

Discovery and trends (2)

Universal video sources (5)

Media assets (5)

Visual search and evidence (3)

Visual search is a dedicated index, separate from transcripts. Every match includes its timestamp, source video, OCR text, and visual description. Local frame paths and file: URLs are removed from MCP results; the browser gallery reads the images locally without exposing those paths to the client.

Comment knowledge base (6)

Diagnostics (3)

Optional API keys

VidLens works without any keys. Add them for scale and reliability:

⚠️ The YouTube and Gemini keys are separate keys from separate Google services. A Gemini key will not work for YouTube API calls, and vice versa.

# Free setup asks for no keys.
npx vidlens-mcp setup

# Enhanced setup optionally prompts for keys, STT, web search, and cookies.
npx vidlens-mcp setup --enhanced

# Or pass everything non-interactively.
npx vidlens-mcp setup \
  --youtube-api-key YOUR_YOUTUBE_KEY \
  --gemini-api-key YOUR_GEMINI_KEY \
  --stt-provider auto \
  --cookies-from-browser chrome

Cookies and speech-to-text

For platforms that rate-limit anonymous access, the wizard can persist cookies by browser profile or file path:

npx vidlens-mcp setup --cookies-from-browser chrome --cookies-profile Default
npx vidlens-mcp setup --x-cookies-file /path/to/x-cookies.txt

Or via environment variables: VIDLENS_COOKIES_FROM_BROWSER, VIDLENS_X_COOKIES_FILE, VIDLENS_INSTAGRAM_COOKIES_FILE, VIDLENS_TIKTOK_COOKIES_FILE.

STT selection is automatic: local whisper.cpp first, then Gemini, then OpenAI. Override with VIDLENS_STT_PROVIDER=whisper-cpp|gemini|openai|none|auto. When the wizard asks, pressing Enter for auto is the right answer for most people.

CLI

npx vidlens-mcp               # Start MCP server (stdio)
npx vidlens-mcp serve         # Start MCP server (explicit)
npx vidlens-mcp setup         # Configure clients, keys, STT, cookies
npx vidlens-mcp doctor        # Run diagnostics
npx vidlens-mcp doctor --no-live   # Diagnostics without network checks
npx vidlens-mcp update-deps   # Refresh managed yt-dlp and Deno helpers
npx vidlens-mcp version       # Print version

doctor checks Node.js version, yt-dlp freshness, STT and web-search providers, API key validity, data directory health, and MCP client registration for Claude Desktop, Claude Code, and Codex.

Requirements

Security and privacy

• Your library — transcripts, frames, embeddings, media — is stored locally and stays on your machine. Network calls go only to the sources you query and the APIs you configure.
• Setup stores configured keys in your local MCP client configuration. Prefer the interactive Enhanced setup, which masks key entry; keys supplied as CLI flags may remain in shell history. Error messages pass through a secret redactor so keys and cookie paths are not echoed in tool output.
• Fetches of channel pages and generic URLs reject private, link-local, and loopback addresses (SSRF guard). Escape hatch for intentional local use: VIDLENS_ALLOW_PRIVATE_URLS=1.

Troubleshooting

"Tool not found" in Claude Desktop. Fully quit (⌘Q) and reopen. MCP servers load only on startup.

"YOUTUBE_API_KEY not configured" warning. Informational, not an error. VidLens works without it.

"API_KEY_SERVICE_BLOCKED". The key is not allowed to call the requested Google service. In Google Cloud Console, enable the YouTube Data API v3 and allow that API in the key's API restrictions. Keep appropriate application restrictions for where you run VidLens; do not make the key unrestricted unless you are only doing a short diagnostic and will restrict it again immediately.

Gemini key doesn't work for YouTube. They are separate services with separate keys. See Optional API keys.

Social video downloads but visual analysis fails. Install ffmpeg, then re-run setup:

brew install ffmpeg

npx vidlens-mcp doctor --no-live

downloadAsset can often fetch a video without ffmpeg, but indexVisualContent, extractKeyframes, media validation, and STT audio processing need ffmpeg or ffprobe.

Anything else:

npx vidlens-mcp doctor

License

MIT

GitHub · npm · Model Context Protocol