Groundhog

Web search, read and research for AI agents — through a real, stealth-patched Chrome. Groundhog is an MCP server that finds pages, reads them, and researches across them, returning clean Markdown a model can trust: text no human could see is stripped by default before the model reads it, every source comes back with a provenance receipt, and a real browser reads pages that block plain fetchers — without the SSRF holes of naive fetch tools.

agent / crawler  ──MCP──▶  Groundhog (search, read_url, research)  ──CDP──▶  stealth Chrome  ──▶  the web

Measured against other fetch layers

Twenty-eight pages, each carrying text a human reader cannot see, plus three controls carrying none. A case passes when the payload does not reach the model and the article text still does — a fetcher that returns nothing contains every payload and is useless.

Groundhog reports 25 of the 28 rather than all of them. The other three — a hidden input value, a meta description, an iframe srcdoc — are contained because no extractor ever collected them, and naming text that was never on its way to the model would describe work nothing did.

Containment happens by accident all the time — an article extractor prunes a hidden <div> because its heuristics dislike it, not because anything asked whether a reader could see it. That is why the last column matters: nothing else tested tells the caller that anything was removed, so an empty threat list and a page with nothing hidden look identical.

The corpus is published at dmytrome.github.io/groundhog, so any fetcher can be measured against the same pages. Harness, per-case results and the limits of what this proves are in benchmark/ — including that hosted services are a snapshot of the date recorded in RESULTS.md.

Quick start

Add Groundhog to your MCP client — that's it. On the first fetch, Groundhog starts the Chrome you already have, headless and in its own profile at ~/.groundhog/chrome: no Docker, no repo checkout, and no window, so it never takes focus while you work. It stays running for the next fetch. Chrome refuses remote control of your everyday profile (since Chrome 136), so your own browsing, tabs and logins are never touched.

Prefer the hardened browser? GROUNDHOG_BROWSER=stealth pulls and starts the stealth-browser container instead (Docker or Podman required). When that path has to run, any stale container named groundhog-browser is removed first; a reachable browser is never touched.

Claude Code:

claude mcp add groundhog -- uvx groundhog-mcp

Claude Desktop / Cursor / Windsurf (claude_desktop_config.json or equivalent):

{
  "mcpServers": {
    "groundhog": {
      "command": "uvx",
      "args": ["groundhog-mcp"]
    }
  }
}

uvx fetches groundhog-mcp from PyPI on first run. The first fetch pulls the browser image (once, a few minutes); later fetches are instant. No container runtime? The status tool and any error say how to install one — or point CDP_URL at a hosted browser for zero-install use.

Prefer to manage the browser yourself? Start it and Groundhog will just use it:

docker run -d --rm --name groundhog-browser --shm-size 512m \
  -p 127.0.0.1:9222:9222 -- ghcr.io/dmytrome/groundhog:latest
# or, from a repo checkout: docker compose up --build -d
curl -s http://localhost:9222/json/version    # CDP is live

Set GROUNDHOG_AUTO_START_BROWSER=false to disable auto-start. To run the MCP server from source: cd mcp && uv sync && uv run groundhog-mcp.

Use any CDP browser

CDP_URL takes the address a browser gives you, used exactly as given. An http:// address is probed for /json/version; a ws:// or wss:// address — what hosted browsers hand out, often with a key in the query — is connected to directly.

CDP_URL=http://127.0.0.1:9222                      # a Chrome you started yourself
CDP_URL="$(agent-browser get cdp-url)"             # a browser agent-browser opened
CDP_URL="wss://<provider host>/?token=<your key>"  # a hosted browser

Auto-start only ever runs for a local address that isn't answering, so a browser you point CDP_URL at is yours to manage. status reports the address without its query or credentials.

All four tools are annotated readOnlyHint, which is what lets a client run them without a per-call confirmation. That describes what they do to your data: nothing is written, and no remote state is changed. Worth knowing, because it is the one exception: with auto-start on, the first call may start Chrome in Groundhog's own profile — or, with GROUNDHOG_BROWSER=stealth, pull and run the browser container and remove an unreachable container named groundhog-browser first. A reachable browser is never touched, and GROUNDHOG_AUTO_START_BROWSER=false turns the whole path off.

What makes it different

• Hidden text is stripped before the model reads it. Groundhog renders a real DOM, so it can judge what a human would actually see and strip what they could not, reporting each occurrence in threats. A strong heuristic, not a proof — see the limits of hidden-text detection, and the benchmark for how that compares. The twelve signals, the threats caveat and the include_hidden exception are documented under read_url.
• Every source carries a receipt. SHA-256 hash of the extracted content, canonical URL, language, word count, and author/date when the page declares them — so a downstream claim traces back to exactly what was read. read_url returns the fetch time alongside it as fetched_at.
• Safe by default. The SSRF guard resolves each host before navigating and refuses to return content from a URL that redirects into a private address. Read-only, with per-domain rate limiting. This matters most in research, where a third party chooses the URLs. See Security for the full blocklist and the guard's limits.
• No automation tell. Puppeteer/Playwright/Selenium enable the CDP Runtime domain, which anti-bots detect (isAutomatedWithCDP). Groundhog drives the browser over raw CDP and never enables Runtime/Console, so that signal is absent — a clean session that full automation libraries can't produce over connect_over_cdp.
• A real fingerprint. It's real Chrome — authentic TLS/HTTP2 fingerprint, real WebGL/canvas — not a Python HTTP client, so fingerprint-driven blocks go away and cheap proxies work where they otherwise wouldn't. Your own Chrome runs headless, and Groundhog gives each tab and each cross-site frame in it the identity of the same Chrome with a window: the HeadlessChrome token is replaced and the client hints are copied from the browser itself, because overriding the user agent alone makes Chrome drop them. Shared and service workers are not covered. The stealth image instead runs Chrome headful under Xvfb.
• No model, no API key. research returns extracts, not summaries; your agent does the synthesis. Self-hosted and MIT — the pages you fetch never leave your infrastructure.

Tools

read_url(url, format="markdown", max_tokens=None, query=None, include_hidden=False)

Fetches a page and returns clean content plus provenance.

Because Groundhog renders a real DOM, it can evaluate computed styles. Text invisible to humans is stripped by default and each occurrence reported in threats with its signal type and a short excerpt: display:none/visibility:hidden, content-visibility: hidden (the subtree is skipped from layout while the element keeps an ordinary box, so no other signal sees it), opacity ≤ 0.05, font-size < 4 px, zero-size elements, an element that generates no box of its own (display: contents) whose contents render nothing, the sub-pixel box used by .sr-only/.visually-hidden accessibility utility classes (a pattern attackers now mimic), the legacy clip: rect(...) hiding technique, fully transparent text color, text color matching the background color (near-1:1 contrast), elements positioned entirely outside the rendered page (e.g. left: -9999px), and content the browser lays out but refuses to render — a collapsed <details>, whose hiding lives on a pseudo-element no per-element style read can see. Non-trivial HTML comments are reported too — they never reach the extracted content either way, but a page embedding instructions this way is worth knowing about. <template> content is treated the same and for a sharper reason: it never renders, so no reader and no computed style sees it, yet it is serialized into the markup the extractor reads, and what a component actually shows is the copy it clones into its shadow tree — which is read from the live page instead. Text carried in an alt, aria-label, aria-description or title attribute is cleared from the markup at any length and reported above a length threshold: no computed style hides it, so the signals above cannot see it, and every image on a page carries an alt — reporting each one would spend the cap on captions. A second, character-level class is stripped and reported alongside these: zero-width characters, bidi marks and RTL overrides, and the Unicode Tag block — an invisible ASCII mirror that is the canonical prompt-injection smuggling channel. Pass include_hidden=True to keep the stripped text in the output; threats is still populated so you know it was there.

Treat threats as untrusted. Entries come in eight shapes (the character classes share one):

The value of stripping is that the payload is out of the content being reasoned over, not that it is invisible to the model. At most 50 findings per page are returned (10 per source in research, since the fan-out multiplies the report); beyond that a report_truncated entry is appended stating how many were dropped, rather than truncating silently. The two classes are capped independently, so a page cannot bury the findings that carry its injection excerpt by flooding the report with decoys of the other kind. Notices are appended after the cap — so they can never themselves be dropped, and a capped list is up to 50 findings plus at most two notices.

Pass query to replace blunt head-truncation with relevance-ranked passage selection: content is chunked on markdown structure, ranked by lexical (BM25) relevance, and the passages scoring within half of the best one are returned, up to the token budget — so a question narrows a page to what answers it rather than to whatever fits the budget, and two different questions about one page come back with different content. matches gives each passage's heading, character offset, and score for downstream citation, and set_aside counts the scoring passages the floor dropped, so a narrowed page is never mistaken for a whole one: truncated still means only that the budget cut the content. Ranking runs on the sanitized content, so hidden-text injection payloads cannot influence which passages surface — with the one exception of include_hidden=True, which leaves the hidden text in the document and ranks it along with everything else.

search(query, limit=10)

Finds pages for a query and returns ranked hits — title, url, snippet, engine, score, published — plus the backend that answered. Hits are links only: nothing is fetched until you pass a URL to read_url.

Two backends, chosen automatically. Set SEARXNG_URL to use your own SearXNG instance (best results; needs formats: [html, json] in its settings.yml, since JSON is off by default upstream). With no instance configured, Groundhog renders a search page through the stealth browser instead — no extra infrastructure, at the cost of depending on that page's layout. Force one with GROUNDHOG_SEARCH_BACKEND=searxng|serp.

Every text field of a hit is attacker-influenceable — a poisoned page controls how it describes itself — so each passes through the same invisible-character stripping as page content, and each is length-capped. The URL is treated differently: it is what a model cites, so it is never rewritten. A hit is dropped outright if cleaning would change its URL at all, if that URL is not http/https, if it carries credentials, or if it exceeds 2048 characters. Both matter on the DuckDuckGo path, which percent-decodes the redirect wrapper and can therefore turn %E2%80%8B back into a real zero-width character inside the link. A backend that is unreachable, has JSON disabled, or whose every upstream engine is rate-limited raises an actionable error rather than reporting an empty web.

research(query, max_sources=5, max_tokens=None)

One call for "find out about X": searches, reads the top sources through the stealth browser, and returns the passages most relevant to query — ranked across all sources in a single pass, so a passage from source 4 competes fairly with one from source 1.

Returns passages (each with text, source_url, heading, score) and sources (each with url, title, status, page_status, threats, provenance). status is the fetch outcome (ok / blocked / timeout / error); page_status is what a page that loaded actually was — the same classification read_url reports, so a source that returned a bot-challenge or a non-HTML body is visible rather than passing as ok — and is null when the fetch never produced a page. A source whose page_status says its body is an interstitial or an error page contributes no passages: it would otherwise compete for your token budget against real content. It still appears in sources, saying why it contributed nothing. At most one page per registrable domain, for source diversity. Passages are extracts, not summaries — nothing is generated, and no model or API key is involved. When a passage isn't enough, read_url its source_url for the whole page.

A source that fails doesn't fail the call: it appears in sources with a status of blocked (SSRF guard), timeout, or error, so a partial answer is still usable and you can see what was missed. Because search results are chosen by a third party — and SEO-poisoned results are a documented in-the-wild attack — every fetched URL goes through the same SSRF guard and hidden-text stripping as read_url, and each source reports what was stripped from it. A source that failed carries provenance: null — only sources that were actually read are hashed. threats is per-source here and capped at 10 entries per source, lower than read_url's 50, because the fan-out multiplies it. max_sources is capped at 10.

It's slower than an API-backed research tool: a real browser renders every source. That's the trade for reading pages that block plain fetchers, and for being able to tell you what was hidden in them.

status()

Reports whether Groundhog can reach the stealth browser. Returns browser_reachable, cdp_url and a hint with remediation steps when it isn't reachable. The endpoint is reported as scheme, host and port only — a hosted browser often carries a credential in its URL, and this value reaches the model.

Configuration

MCP server (mcp/):

Dependencies: py3langid (which pulls in numpy) is used for language detection in the provenance result. It is installed in the MCP server package only — not in the browser container.

Browser container:

Under the hood: the stealth Chrome container

A minimal Docker container running headful Chrome under Xvfb with a remote CDP endpoint. Any CDP-speaking client (Puppeteer, Playwright, Selenium, chromedp, raw DevTools) can drive it — Groundhog is one such client.

• Headful under Xvfb, not --headless=new — the browser reports Chrome, not HeadlessChrome, avoids headless-specific tells, and engages the real GPU path.
• --disable-blink-features=AutomationControlled — navigator.webdriver reads false.
• UA set at launch from the installed Chrome version (USER_AGENT), so it is clean in every scope — main frame, network, and Web/Service Worker globals.
• Proxy geo-coherence. When PROXY is set, the entrypoint geolocates the exit IP and aligns the browser timezone and locale to it — a timezone or locale that disagrees with the IP is itself a block signal. The country→locale table is CLDR likely-subtags. Chrome can't authenticate to a proxy over --proxy-server, so credentials are relayed through a local tinyproxy; WebRTC is pinned to the proxy path so the real IP can't leak.
• GPU-aware WebGL. The entrypoint auto-detects a GPU (NVIDIA via the Container Toolkit, or Intel/AMD via /dev/dri) and uses hardware acceleration; without one it runs Mesa llvmpipe, a coherent software renderer that VMs and servers legitimately emit. See the gpus/devices hints in docker-compose.yml.

Verified results

Measured against a freshly built container (Chrome 151, headful under Xvfb, no proxy), driven over raw CDP:

iphey is tracked informationally, not pass/fail: its one recurring flag is Location ("looks like you're trying to hide your location"), which fires on any datacenter/hosting exit IP regardless of browser fingerprint or TZ correctness — it passes on a residential IP and fails in CI (a cloud runner) and behind most proxies alike.

See RESULTS.md for the full live table (regenerated by tests/antibot.py and the Conformance workflow).

These reflect the raw-CDP client. Full automation libraries (Puppeteer/Playwright/Selenium) enable the CDP Runtime domain and are flagged as automated even against this container — see examples/ for which need patched (rebrowser) variants.

Examples

See examples/OTHER_TOOLS.md for crawl4ai, Scrapy + Playwright, go-rod, Crawlee, and nodriver pointers.

Security

The CDP endpoint is unauthenticated — anyone who can reach the port has full control of the browser. Bind it to localhost or a trusted private network; never expose it to the public internet. The local Chrome listens on 127.0.0.1 only. --no-sandbox is used in the stealth image because Chrome's sandbox does not work in an unprivileged container; keep the container isolated. To report a vulnerability, see SECURITY.md.

Your local network. A local Chrome shares your machine's network, so a page could try to reach services on localhost or your LAN. Groundhog checks the address of every document the browser is about to request — redirects, form posts, script and meta-refresh navigations, and navigations inside cross-site frames, not only the URL it was given — and stops one on a private address before the request is sent. Chrome itself refuses a public page's images, scripts, fetch(), frames and embeds into the local network. The one request it would still make is a speculative prefetch or prerender, so the Chrome Groundhog launches has page preloading turned off. Each of these routes has a live test that fails when its protection is removed. A browser you point CDP_URL at keeps its own preloading setting. GROUNDHOG_BLOCK_PRIVATE_IPS=false turns the document check off.

Limits of hidden-text detection

Worth knowing before treating an empty threats list as a clean bill of health. Nothing is removed from the live page — the markup is stripped inside a separate inert document, which is imported rather than cloned (cloneNode is itself [CEReactions]), and the rendered text comes from the live page with the flagged nodes hidden by an adopted stylesheet. So a page gets no synchronous hook to react to the strip. What that does not cover:

• The style signals are thresholds, and the character set is a denylist. Those are the real limits — see below. The detector itself runs in an isolated world (Page.createIsolatedWorld), so a page cannot suppress it by replacing the DOM APIs it uses; if the browser ever declines to provide one, the result carries a detection_degraded threat rather than quietly weaker detection.
• Thresholds can be sat just inside. opacity: 0.06, font-size: 4px, a contrast ratio just above 1.15 — all pass, as do hiding techniques the twelve signals don't model (clip-path, text-indent, transform: scale(0)).
• Invisible-character coverage is a set, not a rule. Zero-width, bidi and the Unicode Tag block are stripped and reported; codepoints outside that set are not.
• When the text is rebuilt, line breaks are guessed from tag names. In the two cases above the rendered text is taken from the stripped markup, which has no layout — so an element the page styled display:inline still gets a break, and a block-level tag outside the list gets none. Word boundaries are preserved; exact line structure is not.
• Closed shadow roots are not read. Open ones are: their content is scanned for hidden text and composed into the output as the flat tree a reader sees, slots included. A closed root is unreachable from the isolated world, so it cannot be scanned — and what cannot be scanned is not composed in. Its content stays out of the result entirely rather than arriving unexamined.
• A page can win the cascade against the hiding sheet, or hide its own <body>. An inline !important beats an author stylesheet, and innerText returns raw text when nothing renders at all. In either case the rendered text is abandoned for the stripped markup, which is a weaker guarantee than reading real layout — reported as strip_incomplete rather than left to look like a clean strip.

Limits of block detection

status tells you a fetch returned a challenge or an error page rather than the content you asked for. Worth knowing what it does and does not cover.

• Mitigation markers are a list, and the list is not exhaustive. A challenge is called with certainty when the response carries a header that exists only to announce it (cf-mitigated, x-vercel-mitigated, x-amzn-waf-action, x-dd-b, x-datadome-cid) or when the page requests an asset only a challenge loads (Cloudflare's orchestrator, DataDome, PerimeterX, Imperva). A vendor absent from that list, or one that changes its endpoint, falls through to the weaker signals below.
• The wording fallback is English, and only fires on an empty page. A challenge with no recognised marker is caught only if it renders almost no text and matches a known phrase. A localized interstitial from an unlisted vendor is the gap — it is why the markers exist, and why they are preferred over any amount of phrase tuning.
• A soft block is undetectable here. A page that returns 200 with a plausible body but the data quietly withheld looks exactly like content. Nothing in this classification sees it; only comparing against what the page should contain would.
• unknown is not ok. It means no response, or no usable status, was observed for the document that was read. It is reported rather than assumed fine, and it is not treated as a block — a source carrying it still contributes passages in research.
• Vendor presence is not a block. server: cloudflare, cf-ray and __cf_bm are on a large share of the web on every page it serves normally, so they are deliberately not used; nor is cf_clearance, which is issued when a challenge is passed.

What the SSRF guard blocks. Each host is resolved and rejected if it lands in loopback, RFC-1918 private, link-local (incl. 169.254.169.254), reserved, multicast, unspecified, CGNAT 100.64.0.0/10, or IPv4-mapped IPv6 ranges. Only http and https are allowed, and credentials in URLs are rejected. The check runs again immediately before navigation, and once more against final_url after redirects.

Limits of the SSRF guard. It is a strong default, not a sandbox. Know these before pointing it at untrusted URLs:

• The guard resolves and checks the host before navigation and re-checks final_url after the page loads. A redirect into a private address is therefore still requested by Chrome — its content is never returned, but a blind SSRF or a state-changing internal GET has already landed. Intermediate hops in a longer redirect chain are not individually checked.
• Sub-resource requests the page itself issues (img, script, iframe, fetch) are not intercepted; only the top-level navigation is checked.
• Groundhog resolves DNS in its own process while Chrome resolves independently at navigate time, so a short-TTL rebinding window remains open. Closing these properly needs request-level interception (CDP Fetch).
• Fetches share the browser's default profile — targets are created without a separate browser context — so cookies and storage set by one page persist into later fetches. "Read-only" describes Groundhog's own API, not the JavaScript on a fetched page, which can issue requests of its own from that shared profile.

Set GROUNDHOG_BLOCK_PRIVATE_IPS=false only on a network where reaching internal addresses is intended.

A note on "stealth"

Best-effort, not a guarantee. It defeats common open-source detectors and lets cheap proxies work on many mid-tier targets, but it does not beat sophisticated commercial anti-bot systems that gate on IP reputation, TLS/HTTP2 fingerprints, and behavioral analysis. Use it for legitimate, authorized automation and testing.

Privacy Policy

Groundhog is self-hosted software, not a service. It runs on your machine, and the project operates no servers that it talks to.

What is collected: nothing. There is no telemetry, no analytics, no crash reporting and no licence check. The maintainers receive no data about you, the URLs you fetch, or the content you read. There is no account to create.

How data is used and stored. Pages are fetched by a browser running on your own machine and returned to the MCP client that asked for them. The server keeps no database, writes no logs to disk, and persists nothing between calls — with one exception worth knowing: fetches share the browser container's profile, so cookies and storage set by one fetched page remain in that container and are visible to later fetches. Removing the container discards them (docker rm -f groundhog-browser), and the container is removed automatically if you started it with --rm.

Third parties your traffic reaches. Only those you direct it to, plus two you should know about:

• The sites you fetch, which see the request as an ordinary browser visit from your IP — or from your proxy's exit IP if PROXY is set.
• The search backend. With SEARXNG_URL set, your queries go to the SearXNG instance you chose. Without it, search renders a public search engine's results page through the browser, so that engine sees the query.
• An IP-geolocation lookup, only when PROXY is set. The container asks an external service for the proxy exit IP's country so it can align the browser's timezone and locale. That request carries the exit IP and nothing else. It does not happen without a proxy.
• PyPI and GHCR at install time, to download the package and browser image.

Retention. Nothing is retained by the project. On your machine, the browser container holds cookies and cache for its lifetime; deleting the container deletes them.

Contact. Questions and security reports: SECURITY.md, or open an issue at https://github.com/dmytrome/groundhog/issues.

License

MIT