Agent♥︎Age
Catalog

CRW Web Scraper

Official

by fastcrw · Rust

Open-source web scraper for AI agents with scrape, crawl, and map tools

The io.github.fastcrw/crw MCP server is an open-source web scraper for AI agents. It provides tools that support scrape, crawl, and map operations, enabling agent-driven retrieval and traversal of web content for downstream processing.

🛠️ Key Features

  • Open-source web scraper
  • Tools for scrape
  • Tools for crawl
  • Tools for map

🚀 Use Cases

  • Scraping web content for AI agents
  • Crawling through linked pages or resources
  • Mapping web content or relationships based on retrieved data

⚡ Developer Benefits

  • Agent-oriented toolset for web interaction
  • Separate capabilities aligned to scraping, crawling, and mapping

⚠️ Limitations

  • Available details are limited to the server description and do not specify scope, supported sites, rate limits, or configuration options.

Topics

aiai-agentscrawlerdata-extractiondockerhtml-to-markdownllmmarkdownmcprustself-hostedweb-crawlerweb-scraperweb-scrapingfirecrawl-alternativemcp-serverscraping-apitavily-alternativeweb-search-api