Clean raw HTML into LLM-ready text before agents spend tokens.
io.github.LareLabs/refinery-mcp (MCP)
Refinery MCP provides an MCP server wrapper around the Refinery Apify Actor to convert raw HTML—or input URLs—into clean, LLM-ready text. The server is intended to run before agents spend tokens, and it returns cleaned text along with word_count. It supports use with Claude, Cursor, and other agents.
🛠️ Key Features
Wraps the Refinery Apify Actor as an MCP server
Cleans raw HTML into LLM-ready text
Produces word_count alongside cleaned output
🚀 Use Cases
Preparing HTML or URLs for LLM processing
Feeding cleaned content into RAG pipelines
⚡ Developer Benefits
Standardizes HTML-to-text preprocessing via Model Context Protocol
Enables an agent pipeline step: fetch → MCP → clean text → RAG
⚠️ Limitations
Description provided does not specify supported HTML complexity, error handling, or streaming behavior
Refinery MCP wraps the Refinery Apify Actor as an MCP server so Claude, Cursor, and other agents can turn raw HTML or URLs into clean LLM-ready text plus word_count.
flowchart LR
A[Agent needs web context] --> B[Fetch URL or raw HTML]
B --> C[Refinery MCP]
C --> D[Refinery Apify Actor]
D --> E[Clean text + word_count]
E --> F[RAG / embeddings / LLM context]
The Problem
Agents are getting good at fetching web pages. The problem is what they fetch:
html
<html><head><script>gtag("event", "page_view")</script><style>.nav,.cookie,.footer{display:block}</style></head><body><nav>Home · Pricing · Login · Docs · Blog · Careers</nav><aside>Subscribe to our newsletter</aside><article><h1>How ACME cut support ticket routing time by 63%</h1><p>ACME routes 40,000 monthly support tickets through an AI triage system.</p><p>The team reduced retrieval noise by cleaning HTML before chunking.</p></article><footer>Legal · Privacy · Cookie settings · LinkedIn · X</footer></body></html>
The model does not need most of that. It needs this:
text
How ACME cut support ticket routing time by 63%
ACME routes 40,000 monthly support tickets through an AI triage system.
The team reduced retrieval noise by cleaning HTML before chunking.
Before and after: bloated HTML vs clean LLM-ready text and token savings
Refinery MCP gives your agent a tool for that middle step:
text
fetch page -> refine HTML -> send clean text to RAG / embeddings / LLM
Why
Agents can fetch pages, but raw HTML is noisy and expensive:
scripts, styles, tracking tags
nav, footers, cookie banners
repeated links and layout markup
huge token burn before the model sees the real content
Refinery is the middle step your agent can call before it stuffs web context into a prompt:
It is not a crawler. Use Firecrawl, Crawl4AI, Playwright, browser automation, or your own fetcher when you need rendering. Use Refinery when you already have a URL or raw HTML and want a cheap cleanup pass before the LLM.
When To Use It
Use Refinery MCP when:
your agent already fetched a page but got bloated HTML
you want a deterministic cleanup step before RAG ingestion
you need word_count / token-ish savings before embedding
you want to separate crawling from content cleanup
Do not use it as your browser renderer, anti-bot layer, or site crawler.
Tools
clean_url
Fetches a URL through the Refinery Apify Actor and returns dataset rows with clean text and metadata.
Cleans raw HTML your agent, crawler, or browser session already fetched.
Example input:
json
{"html":"<html><body><nav>Home Pricing Login</nav><article><h1>Vendor security update</h1><p>We now support SOC 2 exports for enterprise accounts.</p></article><footer>Legal Privacy Careers</footer></body></html>","extractMentions":false,"extractHashtags":false}
Example result:
json
{"text":"Vendor security update\n\nWe now support SOC 2 exports for enterprise accounts.","word_count":10,"content_type":"web","language":"en","processing_time_ms":44.96,"success":true}
estimate_savings
Local helper that compares raw HTML vs cleaned text and estimates token savings. This does not call Apify.
npm run build
APIFY_TOKEN=apify_api_xxx npm run smoke
The smoke test starts the MCP server over stdio, lists tools, and calls estimate_savings without spending Apify credits.
Example Agent Prompt
text
Use Refinery MCP to clean this docs page before summarizing it:
https://docs.stripe.com/payments
Return the clean text, word_count, and a short summary. Do not summarize raw HTML.
Another useful prompt:
text
I fetched this page HTML with Playwright. Use Refinery MCP clean_html before adding it to my RAG ingestion queue. Return the cleaned text and estimated token savings.