Agentโ™ฅ๏ธŽAge
Catalog

CRW Web Scraper

Official

by fastcrw ยท Rust

Open-source web scraper for AI agents with scrape, crawl, and map tools

The io.github.fastcrw/crw MCP server is an open-source web scraper for AI agents. It provides tools that support scrape, crawl, and map operations, enabling agent-driven retrieval and traversal of web content for downstream processing.

๐Ÿ› ๏ธ Key Features

  • Open-source web scraper
  • Tools for scrape
  • Tools for crawl
  • Tools for map

๐Ÿš€ Use Cases

  • Scraping web content for AI agents
  • Crawling through linked pages or resources
  • Mapping web content or relationships based on retrieved data

โšก Developer Benefits

  • Agent-oriented toolset for web interaction
  • Separate capabilities aligned to scraping, crawling, and mapping

โš ๏ธ Limitations

  • Available details are limited to the server description and do not specify scope, supported sites, rate limits, or configuration options.

Topics

aiai-agentscrawlerdata-extractiondockerhtml-to-markdownllmmarkdownmcprustself-hostedweb-crawlerweb-scraperweb-scrapingfirecrawl-alternativemcp-serverscraping-apitavily-alternativeweb-search-api