Web Metadata, Open Graph & JSON-LD Extractor

Official5 tools

by industrial-platform-ai · JavaScript

Open Graph, JSON-LD, canonical, robots, headings and page metadata extractor via direct x402 MCP.

Model Context Protocol (MCP) Server: io.github.industrial-platform-ai/web-metadata

This MCP server provides x402 “page metadata for agents,” covering Open Graph (OG) tags, JSON-LD, canonical URLs, and robots directives. It is described as charging “$0.01 per URL,” indicating a per-request pricing model. Its primary output is web page metadata fields for agent consumption.

🛠️ Key Features

Generates OG (Open Graph) metadata for agents
Produces JSON-LD metadata
Provides canonical URL data
Provides robots directives
Pricing: $0.01 per URL

🚀 Use Cases

Supplying structured web metadata to agents
Supporting discovery and indexing signals via canonical and robots data
Providing OG and JSON-LD fields for downstream processing

Topics

agent-toolsagentic-aiai-agentsapifyautonomous-agentsdeveloper-toolsmcpmodel-context-protocolresearch-apiweb-research

Captured live from the server via tools/list.

get-actor-run

Get detailed information about a specific Actor run. Returns run result: status, storages (datasets/keyValueStores alias map), stats, summary, nextStep. - summary describes the past (e.g. "SUCCEEDED in 22s. 47 items; 3 fields available."). - nextStep prescribes one primary follow-up action with identifiers interpolated (e.g. "Use get-dataset-items with datasetId=..."). - waitSecs (0–45, default 30) waits up to that many seconds for terminal status before returning. USAGE: - Use to check the status of a run started by any Actor-running tool. - Pass waitSecs > 0 to block until terminal (or until the cap elapses). USAGE EXAMPLES: - user_input: Show details of run y2h7sK3Wc - user_input: Wait for run y2h7sK3Wc to finish This tool requires an x402 payment. Include a valid x402 payment signature in the request metadata (_meta["x402/payment"]). Your MCP client must support the x402 payment protocol.

Parameters2

runId	string	required	The ID of the Actor run.
waitSecs	integer	optional	Maximum seconds to wait for the run to reach a terminal state (SUCCEEDED, FAILED, ABORTED, TIMED-OUT). 0 returns immediately with the current status. Cap: 45. Default: 30.

Raw schema

{
  "type": "object",
  "properties": {
    "runId": {
      "type": "string",
      "minLength": 1,
      "description": "The ID of the Actor run."
    },
    "waitSecs": {
      "default": 30,
      "description": "Maximum seconds to wait for the run to reach a terminal state (SUCCEEDED, FAILED, ABORTED, TIMED-OUT).\n0 returns immediately with the current status. Cap: 45. Default: 30.",
      "type": "integer",
      "minimum": 0,
      "maximum": 45
    }
  },
  "required": [
    "runId"
  ],
  "additionalProperties": false
}

get-dataset-items

Get items (rows) from a dataset — the output/results produced by an Actor run. Returns the rows themselves, not dataset metadata, counts, or a schema. When the user provides a datasetId and asks to retrieve results, output, data, or rows, call this tool directly. Default limit is 20. Use clean=true to skip empty items and hidden fields. USAGE: - Use when you need to read data from a dataset (all items or only selected fields). USAGE EXAMPLES: - user_input: Retrieve results from dataset abc123 - user_input: Get only metadata.url and title from dataset username~my-dataset This tool requires an x402 payment. Include a valid x402 payment signature in the request metadata (_meta["x402/payment"]). Your MCP client must support the x402 payment protocol.

Parameters8

datasetId	string	required	Dataset ID or username~dataset-name.
clean	boolean	optional	If true, returns only non-empty items and skips hidden fields (starting with #). Shortcut for skipHidden=true and skipEmpty=true.
offset	number	optional	Number of items to skip at the start. Default is 0.
limit	integer	optional	Maximum number of items to return. Default is 20.
fields	string	optional	Comma-separated list of fields to include in results. Fields in output are sorted as specified. Use dot notation for nested objects (e.g. "metadata.url"); the server auto-flattens parent prefixes.
omit	string	optional	Comma-separated list of fields to exclude from results.
desc	boolean	optional	If true, results are returned in reverse order (newest to oldest).
flatten	string	optional	Comma-separated list of fields to flatten (e.g. flatten="metadata" turns {"metadata":{"url":"x"}} into {"metadata.url":"x"}). Normally derived automatically from dot-notation in `fields`; specify only as a diagnostic override.

Raw schema

{
  "type": "object",
  "properties": {
    "datasetId": {
      "type": "string",
      "minLength": 1,
      "description": "Dataset ID or username~dataset-name."
    },
    "clean": {
      "description": "If true, returns only non-empty items and skips hidden fields (starting with #). Shortcut for skipHidden=true and skipEmpty=true.",
      "type": "boolean"
    },
    "offset": {
      "description": "Number of items to skip at the start. Default is 0.",
      "type": "number"
    },
    "limit": {
      "description": "Maximum number of items to return. Default is 20.",
      "type": "integer",
      "minimum": 1,
      "maximum": 9007199254740991
    },
    "fields": {
      "description": "Comma-separated list of fields to include in results. Fields in output are sorted as specified. Use dot notation for nested objects (e.g. \"metadata.url\"); the server auto-flattens parent prefixes.",
      "type": "string"
    },
    "omit": {
      "description": "Comma-separated list of fields to exclude from results.",
      "type": "string"
    },
    "desc": {
      "description": "If true, results are returned in reverse order (newest to oldest).",
      "type": "boolean"
    },
    "flatten": {
      "description": "Comma-separated list of fields to flatten (e.g. flatten=\"metadata\" turns {\"metadata\":{\"url\":\"x\"}} into {\"metadata.url\":\"x\"}). Normally derived automatically from dot-notation in `fields`; specify only as a diagnostic override.",
      "type": "string"
    }
  },
  "required": [
    "datasetId"
  ],
  "additionalProperties": false
}

get-key-value-store-record

Get the value stored under a specific key in a key-value store — a single record, not a listing of all keys. Requires the exact key name. The response preserves the original Content-Encoding; most clients handle decompression automatically. USAGE: - Use when you need to retrieve a specific record (JSON, text, or binary) from a store. USAGE EXAMPLES: - user_input: Get record INPUT from store abc123 - user_input: Get record data.json from store username~my-store This tool requires an x402 payment. Include a valid x402 payment signature in the request metadata (_meta["x402/payment"]). Your MCP client must support the x402 payment protocol.

Parameters2

keyValueStoreId	string	required	Key-value store ID or username~store-name.
recordKey	string	required	Key of the record to retrieve.

Raw schema

{
  "type": "object",
  "properties": {
    "keyValueStoreId": {
      "type": "string",
      "minLength": 1,
      "description": "Key-value store ID or username~store-name."
    },
    "recordKey": {
      "type": "string",
      "minLength": 1,
      "description": "Key of the record to retrieve."
    }
  },
  "required": [
    "keyValueStoreId",
    "recordKey"
  ],
  "additionalProperties": false
}

abort-actor-run

Abort an Actor run that is currently starting or running. For runs with status SUCCEEDED, FAILED, ABORTING, ABORTED, or TIMED-OUT, this call has no effect. The results will include the updated run details after the abort request. USAGE: - Use when you need to stop a run that is taking too long or misconfigured. USAGE EXAMPLES: - user_input: Abort run y2h7sK3Wc - user_input: Gracefully abort run y2h7sK3Wc This tool requires an x402 payment. Include a valid x402 payment signature in the request metadata (_meta["x402/payment"]). Your MCP client must support the x402 payment protocol.

Parameters2

runId	string	required	The ID of the Actor run to abort.
gracefully	boolean	optional	If true, the Actor run will abort gracefully with a 30-second timeout.

Raw schema

{
  "type": "object",
  "properties": {
    "runId": {
      "type": "string",
      "minLength": 1,
      "description": "The ID of the Actor run to abort."
    },
    "gracefully": {
      "description": "If true, the Actor run will abort gracefully with a 30-second timeout.",
      "type": "boolean"
    }
  },
  "required": [
    "runId"
  ],
  "additionalProperties": false
}

industrial_platform--web-metadata-intelligence

This tool calls the Actor "industrial_platform/web-metadata-intelligence" and retrieves its output results. Actor description: Use for Open Graph, JSON-LD, canonical, robots, headings, language, and page metadata. Input: urls[]. $0.001 per successful URL; failed URLs are not billed. This tool requires an x402 payment. Include a valid x402 payment signature in the request metadata (_meta["x402/payment"]). Your MCP client must support the x402 payment protocol.

Parameters4

urls	array	required	REQUIRED Public HTTP/HTTPS pages to inspect. Up to 100 unique URLs per run. Example values: ["https://example.com/"]
timeout_seconds	integer	optional	Maximum time allowed for each individual page fetch. Example values: 30
concurrency	integer	optional	Maximum number of URLs fetched concurrently during this run. Example values: 10
waitSecs	integer	optional	Max seconds (0–45, default 30) to cap the wait for the Actor run to reach terminal state. For long-running Actors the response returns at the cap with the current run status; follow `nextStep` to poll via get-actor-run. Set to 0 to fire-and-forget.

Raw schema

{
  "type": "object",
  "properties": {
    "urls": {
      "title": "URLs",
      "description": "**REQUIRED** Public HTTP/HTTPS pages to inspect. Up to 100 unique URLs per run.\nExample values: [\"https://example.com/\"]",
      "type": "array",
      "prefill": [
        "https://example.com/"
      ],
      "items": {
        "type": "string",
        "maxLength": 2048
      },
      "examples": [
        "https://example.com/"
      ]
    },
    "timeout_seconds": {
      "title": "Per-URL timeout",
      "description": "Maximum time allowed for each individual page fetch.\nExample values: 30",
      "type": "integer",
      "default": 30,
      "examples": [
        30
      ]
    },
    "concurrency": {
      "title": "Concurrency",
      "description": "Maximum number of URLs fetched concurrently during this run.\nExample values: 10",
      "type": "integer",
      "default": 10,
      "examples": [
        10
      ]
    },
    "waitSecs": {
      "type": "integer",
      "minimum": 0,
      "maximum": 45,
      "default": 30,
      "description": "Max seconds (0–45, default 30) to cap the wait for the Actor run to reach terminal state. For long-running Actors the response returns at the cap with the current run status; follow `nextStep` to poll via get-actor-run. Set to 0 to fire-and-forget."
    }
  },
  "required": [
    "urls"
  ],
  "title": "Web Metadata Intelligence input",
  "additionalProperties": false,
  "$id": "https://apify.com/mcp/industrial_platform--web-metadata-intelligence/schema.json"
}

README.md13 KBView on GitHub

Industrial Platform Agent Tools

Network trust governance

Industrial Platform uses the Industrial Platform Trust Protocol (IPTP) to separate Base USDC settlement, optional wallet-control verification, independent commercial attribution, and demand-gated factory activation. Identity verification is never a prerequisite to an ordinary x402 purchase. Current implementation is partial; see the protocol for evidence and enforcement boundaries.

Direct recurring x402 MCP

For recurring autonomous workloads, use the direct x402-native MCP first:

text

https://x402-mcp-gateway-production.up.railway.app/mcp

Literal recurring tool names are optimized for auto-import and delegation:

monitor_wallet, transaction_status, watch_transaction, treasury_snapshot, pretrade_context, crypto_candles, crypto_price, crypto_book, market_snapshot, and monitor_webpage_change.

These map to the same canonical paid HTTP workloads, so importer compatibility increases without splitting HTTP settlement history across duplicate routes.

Open-source integration toolkit for connecting AI agents, MCP clients, autonomous workflows, agent orchestrators, and developer applications to Industrial Platform machine-callable services.

Industrial Platform provides specialized paid utilities that other agents can discover and invoke instead of rebuilding research, monitoring, extraction, and document-processing capabilities inside every workflow.

Install in your agent IDE

For the lowest-friction recurring workflow, install Web Metadata + Web Change first. Both currently complete in about 3 seconds median and cost $0.01 per successful unit.

Cursor: one-click install
VS Code / GitHub Copilot: one-click install
GitHub Copilot CLI: copilot mcp add --transport http industrial-platform-fast-web 'https://mcp.apify.com?tools=industrial_platform/web-metadata-intelligence,industrial_platform/web-change-intelligence&payment=x402'
Claude Code: claude mcp add --transport http industrial-platform-fast-web 'https://mcp.apify.com?tools=industrial_platform/web-metadata-intelligence,industrial_platform/web-change-intelligence&payment=x402'
Cline: cline mcp install industrial-platform-fast-web --transport http 'https://mcp.apify.com?tools=industrial_platform/web-metadata-intelligence,industrial_platform/web-change-intelligence&payment=x402'

See INSTALL.md for the full installer matrix, recurring-use patterns, and the full portfolio endpoint.

One MCP endpoint for the full portfolio

text

https://mcp.apify.com?tools=industrial_platform/research-brief-agent,industrial_platform/web-change-intelligence,industrial_platform/web-metadata-intelligence,industrial_platform/article-content-intelligence,industrial_platform/pdf-text-intelligence,industrial_platform/sitemap-intelligence&payment=x402

This hosted MCP endpoint exposes the six currently runnable Industrial Platform services through one direct x402 integration. Tool discovery is free; autonomous callers pay only when invoking a paid Actor.

Current services

Service	Actor	Primary purpose	Billing event	Current price
AI Research Brief	`industrial_platform/research-brief-agent`	Current, cited multi-source research	`research-brief-completed`	$1.99
Web Change Intelligence	`industrial_platform/web-change-intelligence`	Detect deterministic changes in public web content	`page-comparison`	$0.01
Web Metadata Intelligence	`industrial_platform/web-metadata-intelligence`	Extract page metadata, OG/Twitter, JSON-LD, headings	`metadata-extraction`	$0.01
Article Content Intelligence	`industrial_platform/article-content-intelligence`	Extract clean article text for RAG/research pipelines	`article-extraction`	$0.002
PDF Text Intelligence	`industrial_platform/pdf-text-intelligence`	Extract PDF text, per-page text, and metadata	`pdf-extraction`	$0.0015
Sitemap Intelligence	`industrial_platform/sitemap-intelligence`	Discover and expand sitemaps into clean URL rows	`sitemap-url`	$0.0001

Live pricing and execution eligibility should be verified against Apify Store before paid execution.

Official MCP Registry

Industrial Platform is published in the official MCP Registry as:

text

io.github.industrial-platform-ai/industrial-platform

Registry publication is automated on merge.

The Registry listing points to the same hosted six-service Apify MCP bundle used by this repository.

Individual MCP Registry entries

For search-oriented discovery, each paid service also has its own active MCP Registry entry:

Service	MCP Registry name
Research Brief	`io.github.industrial-platform-ai/research-brief`
Web Change	`io.github.industrial-platform-ai/web-change`
Web Metadata	`io.github.industrial-platform-ai/web-metadata`
Article Extractor	`io.github.industrial-platform-ai/article-extractor`
PDF Text	`io.github.industrial-platform-ai/pdf-text`
Sitemap	`io.github.industrial-platform-ai/sitemap`

These individual entries point to the same Apify-hosted Actor tools as the portfolio bundle and improve discovery for capability-specific MCP searches.

Machine-readable discovery

The repository publishes:

text

catalog.json
manifests/mcp-bundle.json
examples/mcp/remote-server.json

Service catalog

catalog.json is the portfolio-level machine-readable catalog. It contains Actor IDs, Store URLs, billing events, current prices, and common delegation targets.

MCP bundle manifest

manifests/mcp-bundle.json defines the hosted MCP endpoint and the Actors included in the bundle.

MCP client configuration

examples/mcp/remote-server.json can be used as a starting point for MCP clients that accept remote server configuration.

Direct x402 framework integrations

Industrial Platform also exposes a direct Base x402 gateway for autonomous callers that do not need the Apify MCP layer:

text

https://x402-gateway-production-1f21.up.railway.app

Machine-readable discovery:

text

https://x402-gateway-production-1f21.up.railway.app/.well-known/x402
https://x402-gateway-production-1f21.up.railway.app/openapi.json
https://x402-gateway-production-1f21.up.railway.app/llms.txt

Direct x402-native MCP:

text

https://x402-mcp-gateway-production.up.railway.app/mcp

The MCP server exposes the same paid tool catalog over Streamable HTTP and performs x402 negotiation at tool-call time. Tool discovery is free; a caller wallet signs the advertised Base USDC requirement when it invokes a paid tool.

Ready-to-copy adapters and workflows are included for:

Minimal official x402 caller — official x402 client, automatic 402 → payment → retry, with a hard per-payment ceiling
Coinbase AgentKit — Base wallet + AgentKit's native x402 provider, including a dry-run-by-default 1-atomic-USDC clean-room canary
ElizaOS — native @elizaos/plugin-wallet x402 client with explicit budget controls and payment disabled by default
LangChain / LangGraph
CrewAI
LlamaIndex
SEO crawler workflow
Crypto intelligence workflow

The examples use the caller's own funded Base wallet, negotiate x402 automatically, and preserve framework-level tool choice rather than forcing Industrial Platform calls.

Agent-to-agent paid quick start

bash

npm install -g @apify/mcpc
mcpc x402 init
mcpc connect "https://mcp.apify.com?tools=industrial_platform/research-brief-agent,industrial_platform/web-change-intelligence,industrial_platform/web-metadata-intelligence,industrial_platform/article-content-intelligence,industrial_platform/pdf-text-intelligence,industrial_platform/sitemap-intelligence&payment=x402" @industrial-platform --x402
mcpc @industrial-platform tools-list

Fund the generated wallet with USDC on Base before paid calls. Product #7 (Link Intelligence) is not included here until its public/agentic publication is confirmed.

Quick start

MCP

Configure one remote MCP server:

json

{
  "mcpServers": {
    "industrial-platform": {
      "url": "https://mcp.apify.com?tools=industrial_platform/research-brief-agent,industrial_platform/web-change-intelligence,industrial_platform/web-metadata-intelligence,industrial_platform/article-content-intelligence,industrial_platform/pdf-text-intelligence"
    }
  }
}

Compatible clients can then discover the exposed Actor tools from that server.

REST / Apify API

Every service can also be invoked directly through Apify's Actor API.

General Actor identifier form:

text

industrial_platform~ACTOR-NAME

Example:

text

industrial_platform~pdf-text-intelligence

Do not commit API tokens, wallet credentials, secrets, or private keys to this repository.

When should an agent delegate?

A calling agent should prefer a specialized Industrial Platform service when delegation is cheaper, simpler, or operationally safer than rebuilding the capability internally.

Examples:

text

Need current multi-source research with citations
→ research-brief-agent

text

Need to know whether a monitored page changed
→ web-change-intelligence

text

Need Open Graph, JSON-LD, canonical, robots, and headings
→ web-metadata-intelligence

text

Need clean article text for a RAG or research pipeline
→ article-content-intelligence

text

Need text and metadata from a public PDF
→ pdf-text-intelligence

Portfolio architecture

text

developer / agent / orchestrator
             ↓
   Industrial Platform MCP
             ↓
  specialized machine service
             ↓
   structured result / event
             ↓
calling system continues workflow

Execution and monetization currently run through Apify.

Authentication and machine payments

Paid execution requires a supported authenticated or agentic-payment path.

Depending on the integration, this may use:

Apify OAuth
Apify API tokens
Apify-supported agentic payment infrastructure

Store eligibility and pricing can change. Calling systems should inspect current Actor metadata before executing paid services.

Repository structure

text

industrial-platform-agent-tools/
│
├── README.md
├── catalog.json
├── AUTONOMOUS_REVENUE_INITIATIVE.md
│
├── actors/
│   ├── web-change-intelligence/
│   ├── web-metadata-intelligence/
│   ├── article-content-intelligence/
│   ├── pdf-text-intelligence/
│   ├── sitemap-intelligence/
│   └── link-intelligence/
│
├── packages/
│   ├── research-mcp/
│   └── change-mcp/
│
├── manifests/
│   └── mcp-bundle.json
│
├── examples/
│   ├── mcp/
│   ├── curl/
│   ├── python/
│   ├── javascript/
│   ├── openai-agents/
│   └── microsoft-agent-framework/
│
└── scripts/
    └── apify/

Revenue initiative

The repository contains automation for:

testing and deployment
Apify runtime configuration
pay-per-event pricing
Store / agentic-payment eligibility
external paid-event telemetry
milestone issues at 1, 10, and 100 external paid events

See:

text

AUTONOMOUS_REVENUE_INITIATIVE.md

Security principles

Industrial Platform services should:

block private/local-network fetch targets when accepting arbitrary URLs
bound response sizes, timeouts, and concurrency
use structured machine-readable outputs
avoid silently fabricating unavailable data
charge successful results rather than failed fetches where practical
keep credentials in secret stores rather than source code

License

Integration examples and open-source code in this repository are provided under the MIT License unless otherwise noted.

Industrial Platform

Organization:

text

industrial-platform-ai

Apify namespace:

text

industrial_platform

Machine-readable catalog:

text

catalog.json

Install

Tools (5)

get-actor-runget-dataset-itemsget-key-value-store-recordabort-actor-runindustrial_platform--web-metadata-intelligence

Remote endpoint

Streamable HTTP

Hosted server - connect over the network, no local install.

url

https://mcp.apify.com?tools=industrial_platform/web-metadata-intelligence&payment=x402

Add to VS Code Add to Cursor

claude_desktop_config.json

json

{
  "mcpServers": {
    "web-metadata": {
      "command": "npx",
      "args": [
        "-y",
        "mcp-remote",
        "https://mcp.apify.com?tools=industrial_platform/web-metadata-intelligence&payment=x402"
      ]
    }
  }
}

Desktop config is stdio-only; this bridges via mcp-remote. Native remote: Settings > Connectors.

Model Context Protocol (MCP) Server: io.github.industrial-platform-ai/web-metadata

🛠️ Key Features

🚀 Use Cases

Topics

get-actor-run

get-dataset-items

get-key-value-store-record

abort-actor-run

industrial_platform--web-metadata-intelligence

Industrial Platform Agent Tools

Network trust governance

Direct recurring x402 MCP

Install in your agent IDE

One MCP endpoint for the full portfolio

Current services

Official MCP Registry

Individual MCP Registry entries

Machine-readable discovery

Service catalog

MCP bundle manifest

MCP client configuration

Direct x402 framework integrations

Agent-to-agent paid quick start

Quick start

MCP

REST / Apify API

When should an agent delegate?

Portfolio architecture

Authentication and machine payments

Repository structure

Revenue initiative

Security principles

License

Industrial Platform

Install

Remote endpoint

Related servers