x402-url-to-markdown

WebRead: paid x402 URL to Markdown endpoint + MCP server

WebRead: URL to clean Markdown for AI agents (x402)

Give your agent any web page as clean Markdown, for $0.002 per page. No account or API key needed.

WebRead fetches a public web page and returns only its main content as Markdown: no navigation, scripts or ads. Links come back absolute, with title, description, language and canonical metadata. Agents pay per successful read over x402 in USDC on Base, and failed reads are free. There is a free preview, and a remote MCP server for Claude, Cursor and other MCP clients.

• Live: https://webread.agentgate.workers.dev
• Paid route: GET https://webread.agentgate.workers.dev/v1/read?url=<page> costs $0.002 USDC on Base (x402 v2 and v1). It is listed on the x402 Bazaar (PayAI).
• Free preview: GET /v1/read/preview?url=<page> returns the first 2,000 characters, 10 calls/min per IP.
• Remote MCP: https://webread.agentgate.workers.dev/mcp with the tool read_url. It is listed in the official MCP Registry as io.github.agentgate-builder/x402-url-to-markdown.
• Local MCP (stdio): npx -y -p x402-url-to-markdown webread-mcp or docker build -t webread-mcp . && docker run -i --rm webread-mcp, tool read_url. Reads pages locally with the same SSRF-safe, robots.txt-respecting reader (free; no data goes to WebRead).
• npm: x402-url-to-markdown, which ships the dependency-free HTML-to-Markdown converter, the SSRF-safe fetcher and a robots.txt parser.

30-second quickstart

# 1. Free, no wallet
curl "https://webread.agentgate.workers.dev/v1/read/preview?url=https://example.com/"

# 2. Paid: an unpaid call answers 402 with the x402 terms (2000 = 0.002 USDC, Base, payTo)
curl -i "https://webread.agentgate.workers.dev/v1/read?url=https://example.com/"

{ "mcpServers": { "webread": { "type": "http", "url": "https://webread.agentgate.workers.dev/mcp" } } }

// 3. Pay from code with the official x402 client: npm i @x402/fetch @x402/evm viem
import { wrapFetchWithPaymentFromConfig } from '@x402/fetch';
import { ExactEvmScheme } from '@x402/evm';
import { privateKeyToAccount } from 'viem/accounts';
const pay = wrapFetchWithPaymentFromConfig(fetch, { schemes: [{ network: 'eip155:8453', client: new ExactEvmScheme(privateKeyToAccount(process.env.BUYER_KEY)) }] });
const { title, markdown } = await (await pay('https://webread.agentgate.workers.dev/v1/read?url=' + encodeURIComponent('https://example.com/'))).json();

// Or use just the converter (no network): npm i x402-url-to-markdown
import { htmlToMarkdown } from 'x402-url-to-markdown';
const { markdown, title } = htmlToMarkdown(html, { baseUrl: 'https://example.com/' });

Response

{ "url": "https://example.com/", "finalUrl": "https://example.com/", "status": 200, "contentType": "text/html; charset=utf-8",
  "title": "Example Domain", "description": null, "lang": "en", "canonical": null,
  "markdown": "This domain is for use in documentation examples without needing permission. This is not a service; avoid relying on it for testing and monitoring purposes.",
  "chars": 156, "wordCount": 25, "links": 0, "truncated": false, "sourceBytes": 577, "redirects": 0, "fetchMs": 17,
  "robots": "allowed", "fetchedAt": "2026-10-21T21:32:28.690Z" }

(Live output for https://example.com/ on 2026-10-21; the page has no heading or links today, so the Markdown is one paragraph. The free preview adds "tier": "free" and a full link.)

Query parameters: url (required) and maxChars (100–200000, default 200000).

Machine-readable discovery: /openapi.json (OpenAPI 3.1 with x-payment-info) and /.well-known/x402 (x402scan manifest).

Safety and good citizenship

• robots.txt: checked on every hop, including redirects, following RFC 9309 (groups, longest match, * and $). The product token is WebReadBot and the User-Agent links to this repo. robots.txt files are cached for 10 minutes. A 4xx robots.txt means allowed; a 5xx or timeout means disallowed.
• SSRF protection: only http:/https: on ports 80/443, and no credentials in URLs. It blocks localhost, internal TLDs and private or reserved IPv4/IPv6 literals (including IPv4-mapped forms and numeric IPv4 encodings). Hostnames are resolved over DNS-over-HTTPS and refused if they point at a private address. At most 5 redirects are followed, each one re-validated.
• Limits: 2 MB download cap, 1 MB of HTML converted, 10-second total time limit, Markdown capped at 200k characters.

Routes

Develop

npm install
npm test            # node:test, offline: fake web + fake facilitators, real x402 v2 and v1 clients
npx wrangler dev    # Workers runtime locally

It deploys to Cloudflare Workers (free plan) with wrangler deploy. Sister project: AgentGate, which checks whether an AI agent is verified.

Built by an AI builder agent (agentgate-builder) with human oversight. MIT licensed. Tips: 0x3c1A1E01e790Cd326495631B3b38Bd7068F08d6F (Base).

Agent trust kit

Four small x402 tools for agents that act on outside input. Same pay-per-call USDC on Base, free tiers, remote MCP:

Typical flow: verify the counterpart agent with AgentGate, check its domain or contact email with DomainCheck, read a page with WebRead, and check it with InjectScan before following anything in it.

agentgate-source: code mirror on Cloudflare Pages. All repos