About the index ↗
MARKET MONITOR / CATALOGUE Services 106,492 MCP listed 39,685 x402 66,861 Rails Base / Solana / Polygon / Arbitrum Imported 2026-10-26 · 3 sources
Task

Robots

48 services carry this tag in their source record. Tags come from the provider, not from us — a tag is a claim about what the service does, not proof that it does it well.

Saylor Agent Tools x402 REST usage

TLS certificate check for a domain - issuer, subject, validity window, days remaining and fingerprint

saylorinnovations.com Web & Scraping 3 calls / 30d checked 2026-10-26
0.01 USDC per call per call

Page Links x402 REST usage

Fetch a reviewed public page and return its links with resolved absolute URLs and anchor text, the canonical URL, the declared meta robots directive, and how many links leave the host. Relative hrefs are resolved against...

api.zfinia.com Web & Scraping 6 calls / 30d checked 2026-10-26
0.004 USDC per call per call

Robots.txt check x402 REST usage

Robots.txt check. Read a site's robots.txt and say whether a crawler may fetch a URL, by RFC 9309 rules as Google applies them (most specific user-agent group, longest matching Allow or Disallow, Allow on ties, * and $ w...

agent-data-tools.revmesh2074.workers.dev Geo & Maps 2 calls / 30d checked 2026-10-26
0.01 USDC per call per call

Delx Robots Inspect x402 REST usage

Robots.txt inspection API for agents at $0.01 USDC. Call when you need crawl rules and sitemap hints before indexing or scraping a domain. Use for SEO preflight and polite crawlers via x402—no API key, no subscription.

api.delx.ai Crypto & Payments 2 calls / 30d checked 2026-10-26
0.01 USDC per call per call

Crawl Pre-Flight x402 REST usage

Crawl pre-flight for a domain: will you be blocked, is content behind a paywall, and how much is actually there. Returns per-bot robots.txt verdicts for 16 AI crawlers, crawlable page count from sitemaps, detected CDN an...

crawl-preflight.postnov01.workers.dev Geo & Maps 1 calls / 30d checked 2026-10-26
0.02 USDC per call per call

robots.txt & AI crawler rules x402 REST usage

Parse a site's robots.txt: whether a user-agent may crawl a path, which AI crawlers (GPTBot, ClaudeBot, Google-Extended, PerplexityBot, CCBot...) are blocked, and declared sitemaps. Check before your agent scrapes.

api.vextorium.com Web & Scraping 5 calls / 30d checked 2026-10-26
0.003 USDC per call per call

AveDaris Agent Utilities x402 REST usage

Check a UUID: send a string and get whether it is a valid UUID plus its version and variant. $0.001 USDC per call on Base.

api.avedaris.com Crypto & Payments 14 calls / 30d checked 2026-10-26
0.001 USDC per call per call

Robots.txt check x402 REST usage

Fetch a site's robots.txt and answer: may this user-agent crawl this path? Returns the matched rule and all declared sitemaps.

agent402.tools Search & Research 7 calls / 30d checked 2026-10-26
0.001 USDC per call per call

Robots.txt & Sitemap x402 REST usage

A site's robots.txt rules per user agent, whether AI crawlers (GPTBot, ClaudeBot, Google-Extended, CCBot, PerplexityBot) are blocked, and the URLs listed in its sitemap. Use before crawling a site or for SEO audits.

bess-x402.onrender.com Geo & Maps 2 calls / 30d checked 2026-10-26
0.002 USDC per call per call

Forest Web x402 REST usage

Convert one bounded fiat amount using the latest successfully fetched European Central Bank daily foreign-exchange reference table. Forest returns the ECB observation date and deterministic cross-rates through EUR. Fresh...

http--forest-gas-station-mainnet--lcjl27p8lmjs.code.run Crypto & Payments 3 calls / 30d checked 2026-10-26
0.001 USDC per call per call

wodan-posture x402 REST

Sources report for one URL (sitemaps, feeds, llms.txt, endpoints), as an MCP tool.

wodan-posture.app.goedvps.com AI Models & Inference checked 2026-10-26
no price published

ai-crawler-policy x402 REST

AI-crawler policy of a website: reads robots.txt and reports, for 20 known AI/LLM crawlers (GPTBot, ChatGPT-User, OAI-SearchBot, ClaudeBot, anthropic-ai, CCBot, Google-Extended, Applebot-Extended, PerplexityBot, Bytespid...

intel.rallylive.ca AI Models & Inference checked 2026-10-26
0.01 USDC per call per call

robots-allowed-for-bot x402 REST

Is a specific bot allowed to fetch a URL? Fetches the site's robots.txt and evaluates the path for the named user-agent (GPTBot, ClaudeBot, Googlebot, Bingbot, CCBot, PerplexityBot or any token), returning allowed/disall...

intel.rallylive.ca AI Models & Inference checked 2026-10-26
0.01 USDC per call per call

meta-tag-robots x402 REST

The robots meta tag of a page: whether it is present and its content (name= or property= form), plus length and a note when the value looks wrong (empty, too long, or a placeholder). One tag, one answer, for SEO and soci...

site.intel.rallylive.ca Business & Growth checked 2026-10-26
no price published

meta-tags-robots-watch x402 REST

Change watch for meta tags robots: runs the check, compares it with the result from your previous call for the same url (kept 90 days) and returns the current answer, changed true/false, the fields that changed with befo...

page.intel.rallylive.ca Web & Scraping checked 2026-10-26
no price published

meta-tags-robots x402 REST

All robots meta tags of a page as a name → content map, with a count and a list of expected tags that are missing (for Open Graph/Twitter). One group per call for social-preview and SEO checks. $0.01 per page.

page.intel.rallylive.ca Business & Growth checked 2026-10-26
no price published

robots-txt-parser x402 REST

robots.txt fetched and parsed: every user-agent group with its allow/disallow rules and crawl-delay, declared sitemaps, and whether the file exists at all. Understand a site's crawl policy before fetching it. $0.01 per s...

intel.rallylive.ca Search & Research checked 2026-10-26
0.01 USDC per call per call

crawl-allowed-check x402 REST

Is a URL crawlable? Fetches the site's robots.txt and evaluates the given path for a specific user-agent (Googlebot, GPTBot, ClaudeBot, Bingbot or any token; default *), returning allowed/disallowed, the matching rule, t...

intel.rallylive.ca Search & Research checked 2026-10-26
0.01 USDC per call per call

Robots check x402 REST

Checks whether a given user agent may fetch a given path on a host, per the site's robots.txt (specific-agent groups, wildcard rules, Allow/Disallow longest-match). Also returns crawl-delay, sitemap URLs and whether the...

relay402.georgespring.workers.dev Search & Research checked 2026-10-26
0.001 USDC per call per call

x402 Funnel Conversion Audit x402 REST

Paid x402 funnel conversion audit (cycle 201, x402 $0.001) - answers the only question 200 cycles of seller-side audits never asked: does a seller CONVERT the crawls it already gets? Reads the seller's OWN access log, se...

6xrkjcfa3yrzvefcpvanurzou4.srv.us Crypto & Payments checked 2026-10-26
0 USDC per call per call

x402 Intent Signal Extractor x402 REST

Paid x402 intent signal extractor (cycle 201, x402 $0.001) - ranks a seller's catalog against the intents a buyer agent arrives with. Reads the seller's llms.txt + manifest, extracts declared intents, token-matches each...

6xrkjcfa3yrzvefcpvanurzou4.srv.us Crypto & Payments checked 2026-10-26
0 USDC per call per call

Ai Agent Readiness Card x402 REST

Paid AI-agent readiness card composer (cycle 157, x402 $0.001) - produces a single structured card telling an AI agent whether a target URL is safe+affordable to call. Scores 6 axes (each 0-100) with equal weight: llms.t...

6xrkjcfa3yrzvefcpvanurzou4.srv.us Crypto & Payments checked 2026-10-26
0 USDC per call per call

CSP Permitted Script Mapper x402 REST

Paid CSP x Live-Script mapper (cycle 156, x402 $0.001) - fetches target HTML, extracts every <script src=...> URL, and maps each against the target's declared CSP script-src + default-src allow-list. Reports which runnin...

6xrkjcfa3yrzvefcpvanurzou4.srv.us Geo & Maps checked 2026-10-26
0 USDC per call per call

Web Bot Auth Verify x402 REST

Paid Web Bot Auth / RFC 9421 HTTP Message Signatures verifier (cycle 146, x402 $0.001) - probe /.well-known/http-message-signatures on target, extract Ed25519 public keys, parse caller-supplied signature-input + signatur...

6xrkjcfa3yrzvefcpvanurzou4.srv.us Crypto & Payments checked 2026-10-26
0 USDC per call per call

Related tasks