clean-markdown-mcp
An MCP server that turns any URL into clean,
LLM-ready Markdown — no ads, no nav bars, no cookie banners, no scripts.
Built for RAG pipelines and AI agents, where junk in the input means junk in the
output.
Give it: https://en.wikipedia.org/wiki/Markdown
Get back: # Markdown
Markdown is a lightweight markup language for creating
formatted text using a plain-text editor...
Install
npm install -g clean-markdown-mcp
Use it with Claude Desktop
Add this to your claude_desktop_config.json:
- macOS:
~/Library/Application Support/Claude/claude_desktop_config.json
- Windows:
%APPDATA%\Claude\claude_desktop_config.json
{
"mcpServers": {
"clean-markdown": {
"command": "npx",
"args": ["-y", "clean-markdown-mcp"]
}
}
}
Restart Claude Desktop, then ask it to "scrape https://example.com to Markdown."
Works the same way in Cursor, Windsurf, or any other MCP client — point it at the
clean-markdown-mcp command.
scrape_url
| Parameter | Type | Description |
|---|
url | string | The page to scrape. Must be http:// or https://. |
includeLinks | boolean | Keep hyperlinks in the Markdown. Default true. Set false for cleaner prose. |
renderJs | boolean | Load the page in a real browser first, for JavaScript-built sites. Default false. Requires Playwright — see below. |
Returns the page title, source, word count, and the clean Markdown.
How the cleaning works
- Fetch — with a 20-second timeout, a 5 MB size cap, and redirects validated
at every hop.
- Clean — runs Mozilla Readability,
the engine behind Firefox Reader View, to isolate the real article and discard
navigation, sidebars, ads, and comment sections.
- Convert — Turndown renders it as
GitHub-flavoured Markdown, preserving headings, lists, tables, and code blocks.
If a page isn't article-shaped, it falls back to a cleaned <body> rather than
failing, so you still get usable text.
JavaScript-heavy sites (optional)
Some sites build their content with JavaScript after loading. A plain fetch
returns almost nothing for those. To handle them, install Playwright once:
npm install playwright && npx playwright install chromium
Then pass renderJs: true. The difference on such a page is dramatic:
| Mode | Result |
|---|
| Default | ~3 words — just navigation links |
renderJs: true | ~190 words — the full content |
Playwright is not a dependency of this package, so a normal install stays
small and fast. Without it, renderJs returns a clear message telling you how to
enable it.
Security
The scraper refuses private, loopback, and link-local addresses — including cloud
metadata endpoints like 169.254.169.254 — and re-validates every redirect hop.
This matters because an MCP server runs on your machine, with access to your
local network.
Need to scrape at scale?
This package handles one page at a time, locally. For batch scraping,
hosted JavaScript rendering, and a pay-per-page API with no infrastructure to
run, the same engine is available as a hosted Actor:
https://apify.com/perforated_hummingbird/url-to-markdown
Known limitations
- Some sites block automated traffic. That's a site policy, not a bug here.
- Non-UTF-8 pages may render with garbled characters.
- No
robots.txt enforcement — you are responsible for how you use it.
Local development
npm install
npm run build
node dist/try.js https://example.com
node dist/try.js https://example.com --render
License
MIT