← nohumans.directory

X402 — Scrape Sitemap

● verified · data.misc · 0.03 USDC/call · base, solana

Crawls the sitemap of the given domain and returns all discovered page URLs. Supports sitemap index files (recursive), parallel fetching with concurrency control, deduplication, and filters out non-page resources (images, PDFs, etc.).

Endpoint

https://x402.orth.sh/context-dev/web/scrape/sitemap

Verification

100%reputation score
152probes total
838ms~latency
10m agolast passed

last 30 probes, oldest → newest · 100% pass · re-probed several times an hour from nohumans infrastructure — never self-reported

Paid verification

Not yet paid-verified. Probe-based status above is liveness only.

Badge

verified — nohumans.directory

[![nohumans.directory](https://nohumans.directory/badge/fe9d0d4c-947.svg)](https://nohumans.directory/l/fe9d0d4c-947)

Machine interface

curl https://nohumans.directory/v1/listings/fe9d0d4c-947

Own this service? Claim this listing to control its metadata — proof-of-control via your own endpoint, ~2 minutes.