10 services carry this tag in their source record. Tags come from the provider, not from us — a tag is a claim about what the service does, not proof that it does it well.
Apify is the largest and most trusted marketplace of tools for web scraping, crawling, data extraction, and automation. Use these tools, called Actors, to extract structured data from social media, e-commerce sites, sear...
Email deliverability & domain security check — validates SPF, DKIM (multi-selector), and DMARC, detects spoofing and misconfigurations, and returns an A–F email security grade with transparent deductions. Anti-spam / ant...
List the URLs of a site's XML sitemap (found via robots.txt if needed, following sitemap indexes and .gz files) with last-modified dates, optionally filtered by text. Discover a site's pages for crawling or monitoring.
All links on a page classified as internal or external, with anchor text, nofollow/sponsored, new-tab flags and the most linked external domains. For SEO, crawling and audits.
Fetch and parse a sitemap.xml (or sitemap index): returns up to 500 URLs with lastmod, or the child sitemaps of an index.
Use before spending tokens, money or crawler capacity fetching an unfamiliar web resource. Resolves redirects (manually, bounded, re-validated per hop) and returns deterministic HTTP/resource facts — final URL, status, c...
robots.txt parser: rules, sitemaps, AI-crawler blocks, and is-this-path-allowed check — Genesis402 / UnyKorn Operator Network
Fetch a site's robots.txt and answer: may this user-agent crawl this path? Returns the matched rule and all declared sitemaps.
Sitemap reader: list a site's URLs with lastmod (handles sitemap indexes) — Genesis402 / UnyKorn Operator Network
A site's robots.txt rules per user agent, whether AI crawlers (GPTBot, ClaudeBot, Google-Extended, CCBot, PerplexityBot) are blocked, and the URLs listed in its sitemap. Use before crawling a site or for SEO audits.