← Audit Tools · All best-of audits
Best Website crawler for AI and LLMs on Apify - July 2026 Audit
Audited 2026-07-24 · metadata scan (live platform signals, no marketing copy) · methodology
Winner today: Website Content Crawler at 96/100. This page is regenerated from a fresh audit on the date above. Rankings move when actors improve or rot. Do not trust a listicle older than its data: rerun this audit yourself with the command at the bottom.
The ranking
Candidates are the top results for "website content crawler" on the Apify store at audit time, scored on measured signals: 30-day reliability, maintenance freshness, permission posture, pricing transparency, publisher health, and more. All criteria.
The receipts
Website Content Crawler - 96/100
- Pricing Transparency: 100 - Pricing model: PAY_PER_EVENT.
- Marketplace 30-Day Reliability: 97 - 146 succeeded / 150 total runs in last 30 days (97.3% success rate). Failed: 1, Timed out: 0, Aborted: 3.
- Rot Check: 100 - Last modified 2 days ago (2026-07-21).
- Listing Freshness: 100 - Last modified 2 days ago.
Website Content Crawler - 87/100
- Pricing Transparency: 100 - Pricing model: PAY_PER_EVENT.
- Marketplace 30-Day Reliability: 100 - 42 succeeded / 42 total runs in last 30 days (100.0% success rate). Failed: 0, Timed out: 0, Aborted: 0.
- Rot Check: 100 - Last modified 92 days ago (2026-04-23).
- Listing Freshness: 83 - Last modified 92 days ago.
Fast Website Content Crawler - 85/100
- Pricing Transparency: 100 - Pricing model: PAY_PER_EVENT.
- Marketplace 30-Day Reliability: 100 - 45496 succeeded / 45519 total runs in last 30 days (99.9% success rate). Failed: 13, Timed out: 0, Aborted: 10.
- Rot Check: 90 - Last modified 162 days ago (2026-02-12).
- Listing Freshness: 65 - Last modified 162 days ago.
Deep Website Content Crawler - 85/100
- Pricing Transparency: 100 - Pricing model: PAY_PER_EVENT.
- Marketplace 30-Day Reliability: 100 - 7347 succeeded / 7353 total runs in last 30 days (99.9% success rate). Failed: 5, Timed out: 0, Aborted: 1.
- Rot Check: 92 - Last modified 148 days ago (2026-02-26).
- Listing Freshness: 69 - Last modified 148 days ago.
Website Content Crawler - 76/100
- Pricing Transparency: 90 - Pricing model: FREE.
- Marketplace 30-Day Reliability: 97 - 2227525 succeeded / 2289650 total runs in last 30 days (97.3% success rate). Failed: 9120, Timed out: 39637, Aborted: 13368.
- Rot Check: 100 - Last modified 0 days ago (2026-07-24).
- Listing Freshness: 100 - Last modified 0 days ago.
Rerun this audit yourself
Every score above is reproducible right now. No account needed for the free metadata mode:
curl -X POST https://audit-tools.ai/api/evaluations \
-H "Content-Type: application/json" \
-d '{"targets": ["apify/website-content-crawler","6sigmag/fast-website-content-crawler","parseforge/website-content-crawler","6sigmag/deep-website-content-crawler","alizarin_refrigerator-owner/website-crawler"], "mode": "metadata"}'
AI agents can pay per report with USDC via x402 or use the MCP setup in the docs. See the API reference.
Generated by Audit Tools: measured, not marketed. Scores where our own products lose stay published.