# Audit Tools, Full LLM Context

Last updated: 2026-07-28

> Given a scraping task, which Apify actor should you actually use? Audit Tools answers that from measurements instead of listing copy: public platform data and live probes on real inputs, with the evidence attached to every score. Agent-native via x402 and USDC.

URL: https://audit-tools.ai
Not affiliated with Apify. Independent evaluation layer.

---

## What Audit Tools does

Audit Tools evaluates Apify Store actors empirically. You give it actor IDs, it returns evidence-backed scores for each criterion. Up to 5 actors side by side in one run, across the full public catalog of about 41,000 actors (40,995 counted live on the site on 2026-07-28), not a curated subset.

The core claim: always run the best actor available today, not the one hardcoded six months ago.

Free modes cost nothing and need no account: `metadata` and `estimate` both return a scored report from live platform data. The deep scan adds the criteria that require executing the actor. It is $1.00 flat per evaluation, by card on the site or over x402.

## Why a measurement layer is needed

- A Store listing states claims, not current behavior. Advertised fields frequently come back null.
- Actors reprice without notice. In one audited week, two actors in a single category changed price.
- The same job can differ 500x in cost between actors with near-identical scores.
- "Zero failed runs" can hide hundreds of timeouts, because failures and timeouts are counted separately.
- Popularity is not reliability. In one audited suite the most-reviewed actor, with 227 reviews, scored 5 points below a sibling with 33 reviews.

---

## API, how agents call Audit Tools

Machine-readable spec: https://audit-tools.ai/openapi.json

### Create an evaluation

POST https://audit-tools.ai/api/evaluations

Request body (JSON):
```json
{
  "targets": ["apify/instagram-scraper", "clockworks/tiktok-scraper"],
  "mode": "metadata",
  "criteriaIds": ["rot_check", "marketplace_reliability", "x402_payability"],
  "customCriteria": [
    {
      "id": "my-criterion",
      "name": "My custom criterion",
      "rubric": "Actor returns structured data with price, title, and availability fields populated",
      "weight": 1.0
    }
  ]
}
```

Fields:
- targets (required): array of Apify actor IDs. Both username/actorname and username~actorname work, and a full Apify Store URL is accepted too - it is normalised to the actor id. 1 to 5 targets for the free modes, exactly 1 for the paid deep scan.
- mode: "metadata" or "estimate" are both free and return the same scored report. "full" is the $1.00 deep scan, which adds the criteria requiring a live actor run. Default: "metadata".
- criteriaIds: array of criterion IDs to evaluate. Omit it and the free modes run the 13 defaults, while the paid deep scan runs all 23 available criteria.
- customCriteria: your own criteria described in plain English, scored against actor metadata and README by an LLM judge with evidence. The optional weight on each one is how heavily your criterion counts in the overall score.

Response:
```json
{
  "id": "eval-uuid",
  "status": "complete",
  "mode": "metadata",
  "targets": [
    {
      "actorId": "apify/instagram-scraper",
      "actorName": "instagram-scraper",
      "actorTitle": "Instagram Scraper",
      "status": "complete",
      "overallScore": 95,
      "scores": [
        {
          "criterionId": "rot_check",
          "name": "Rot Check",
          "score": 100,
          "status": "measured",
          "passed": true,
          "evidence": "Last modified 0 days ago (2026-07-28)."
        }
      ]
    }
  ],
  "createdAt": "2026-07-28T13:00:00.000Z"
}
```

Notes an agent needs:
- A criterion that cannot be measured is returned with status "unmeasured" rather than a guessed number.
- A target that does not exist on the Apify Store returns status "not_found" with a machine-readable errorCode of TARGET_NOT_FOUND, and no score.
- Every score carries an `evidence` string describing what it was derived from.

### Get one evaluation report

GET https://audit-tools.ai/api/evaluations/{id}

### List recent public evaluations

GET https://audit-tools.ai/api/evaluations/recent

### List available criteria

GET https://audit-tools.ai/api/criteria

### Live Apify Store catalog count

GET https://audit-tools.ai/api/store/count

### Paid deep scan and x402

The deep scan answers an unpaid request with 402 Payment Required and an x402 payment envelope: USDC on Base, $1.00 flat, settled through the Coinbase facilitator. Audit Tools is listed in Coinbase's public x402 directory, so an agent can discover the endpoint and its price programmatically. Any x402-capable client works.

### Running it inside your own Apify account

The same evaluation is published as an Apify actor, so it can run inside your own Apify account and pipelines. Apify meters it per event: $0.00001 per metadata evaluation, a cent for a thousand, and the same $1.00 for a deep scan. Calling audit-tools.ai over HTTP stays free for the metadata and estimate modes, so the actor is for teams that want the audit to run where their other scraping runs already live.

---

## Criteria catalog

Any criterion you can describe in plain English can be scored, with no cap. The 23 built-in criteria, of which 13 run by default, are a head start, not a limit. Of the 13 defaults, 8 are measured at no cost from platform data and 5 require a funded live run.

### Measured at no cost, from live platform data

**rot_check** (default)
Actor still exists (200 against 404), plus last build and modify date. A 404 is full rot.

**marketplace_reliability** (default)
Success rate from public 30-day run outcome counts: succeeded divided by total runs, read in the context of run volume.

**listing_freshness** (default)
Days since last modification. Recently updated actors are less likely to be broken by upstream changes.

**pricing_transparency** (default)
Actor has a defined, public pricing model rather than UNKNOWN. Pay-per-event is the most transparent and the most agent-compatible.

**publisher_pulse** (default)
Publisher portfolio health: ratings, failure rates and build activity across all of their actors.

**x402_payability** (default)
Whether an autonomous agent can call and pay for this actor with no human step: whitelisted for agentic payments, pay-per-event pricing, limited-permission compatible.

**permission_posture** (default)
Limited against full permission. Full-permission actors need human console approval, which blocks autonomous execution entirely.

**user_trajectory** (default)
Monthly against total users: growing, flat or dying adoption.

**schema_validity**
Actor declares an output schema with defined fields. Undeclared schemas are opaque to automated pipelines.

**machine_readable_contract**
Stable JSON schema, typed fields and deterministic error codes, the contract an agent needs to call an actor reliably without human interpretation.

**adoption_momentum**
Total users and weekly active users. A high total with no weekly activity suggests abandonment.

**input_schema_completeness**
A typed input schema with required fields, so an agent does not have to guess the input structure.

**input_schema_documentation**
Input fields carry titles and descriptions, so an agent can determine correct values without reading source code.

**readme_length**
Documentation depth. A sub-500-character README signals an unmaintained or publisher-only actor.

**readme_has_examples**
At least one concrete usage example: curl, a JS or Python snippet, or sample input. Critical for autonomous integration.

**run_history_depth**
Whether the actor has meaningful run history. Zero total runs means untested in production by anyone.

**quality_delta**
Score change since the previous audit of this actor. A drop of 10 or more points raises a regression flag.

### Require funding

**advertised_field_truth** (default)
The specific fields the listing advertises exist and are populated in the output. The most common form of silent failure.

**cost_per_item** (default)
Metered cost divided by the items that pass the caller's quality bar. An actor that costs more per run but returns three times the usable results can still win on cost per result.

**end_to_end_latency** (default)
Wall-clock time from request to usable dataset on real input, observed rather than estimated.

**run_consistency** (default)
Same input twice: overlap of results, and stability of shape and count. High-variance actors are unreliable in pipelines.

**memory_fit** (default)
Completes on the default memory allocation, or OOMs and demands paid upsizing, which is hidden cost.

**readme_structure**
Organized sections, input and output tables, troubleshooting. Needs no actor run, but an LLM judge reads the README against a rubric, so it is metered separately.

---

## How Audit Tools differs from the alternatives

Store search and popularity trackers answer "what exists" and "what is popular". Recommender lists built on curated subsets answer "what someone shortlisted". None of them answer "which of these actually works for my task right now, and what will it truly cost me".

| | Audit Tools | Store search | Popularity trackers | Curated recommender lists |
|---|---|---|---|---|
| Catalog | Full public store, about 41,000 actors | Keyword index of the store | Tracked subset | Curated shortlist |
| Answers the selection question | Yes | No | No | Partly |
| Live probe execution | Yes | No | No | No |
| True cost per usable result | Yes | No | No | No |
| Advertised-field truth check | Yes | No | No | No |
| Custom criteria in plain English | Yes | No | No | No |
| Agent-native payment (x402) | Yes | No | No | No |
| Evidence attached to every score | Yes | No | No | No |
| Data freshness | At evaluation time | Unknown | Periodic | Last catalog update |

---

## Research & blog posts

### Apify YouTube Transcript Actors, July 2026 Audit (published 2026-07-28)

URL: https://audit-tools.ai/blog/youtube-transcript-actors-july-2026.html

An Apify pricing-change notice arrived on July 28, 2026 for one YouTube transcript actor. Rather than audit one vendor, we audited the eight most-used YouTube transcript actors on the Store the same afternoon, in free metadata mode, because eight actors doing an identical job is the sharper comparison.

**Audit scores (July 28, 2026), with free-plan cost per 1,000 transcripts and 30-day success rate:**
- YouTube Transcript Scraper Pro (codepoetry/youtube-transcript-ai-scraper): 97/100 · $1.00 · 97.8% · highest score, no reviews yet
- YouTube Video Transcript (starvibe/youtube-video-transcript): 96/100 · $5.00 · 100.0% across 492,138 runs · 5.00/5 from 20 reviews · most reliable, best pick for volume
- Youtube Transcript Scraper (supreme_coder/youtube-transcript-scraper): 96/100 · $1.00 free plan, $0.70 paid · 99.0% · the actor that was repriced
- YouTube Transcript & Subtitles Scraper API (johnvc/YoutubeTranscripts): 96/100 · about $0.02 · 95.5% · cheapest by 500x
- Youtube Transcripts (karamelo/youtube-transcripts): 95/100 · $7.00 · 99.9%
- YouTube Transcript Ninja (topaz_sharingan/Youtube-Transcript-Scraper-1): 95/100 · $10.00 · 99.7%
- Youtube Transcript Scraper (pintostudio/youtube-transcript-scraper): 88/100 · $10.00 · 99.7% · stalest at 145 days
- Best Youtube Transcripts Scraper (scrape-creators/best-youtube-transcripts-scraper): 88/100 · $1.00 · 85.0% with 6,659 failed runs · last place despite the name

**The pricing change in full (supreme_coder/youtube-transcript-scraper):** actor start went $0.001 to $0.00005, which is 95% cheaper. Per transcript went $0.0005 to $0.0007 on paid plans, 40% dearer, and to $0.001 on the free plan, 100% dearer. So a 1-transcript run got 50% cheaper while a 1,000-transcript run went from $0.501 to $0.70005 on paid plans. The crossover is 5 transcripts per run on paid plans and 2 on the free plan. The listing advertises $0.7 per 1k, which is the paid-plan rate only.

**Key findings:**
- The same job costs $0.02 to $10.00 per 1,000 transcripts, a 500x spread, and price predicted neither reliability nor freshness.
- Zero failed runs is not zero lost runs. Two actors logged 0 failures but hundreds of timeouts, which only appears if you read succeeded divided by total runs.
- The actor with "best" in its name finished last on reliability (85.0%) and on rating (4.14/5). A name is marketing.
- Two of the eight repriced within one week, so any hardcoded actor choice goes stale.

### An AI Agent Bought One of Our Audit Reports via x402 (published 2026-07-25)

URL: https://audit-tools.ai/blog/ai-agent-bought-an-audit.html

On July 24, 2026 an AI agent paid $1.00 in USDC over x402 and received a full audit report. No human, no account, no credit card. It has happened four times, most recently on July 29, 2026; all four payments were made by our own test agent, disclosed in the post, and are verifiable on the public blockchain.

**Key facts:**
- Audit Tools is listed in Coinbase's public x402 directory, so any agent can discover the API and its price programmatically.
- The paid endpoint returns 402 with an x402 envelope: USDC on Base, $1.00 flat, settled via the Coinbase facilitator.
- Any x402-capable client works; with x402-fetch it is three lines of code.
- Payment proof: https://www.x402scan.com/recipient/0x95de884E9eb3F90E496200693d94a636942a130D
- Free evaluations stay free. Only the deep scan costs $1.00.

### TikTok Apify Actors, June 2026 Price Change (published 2026-06-30)

URL: https://audit-tools.ai/blog/tiktok-actors-june-2026.html

On June 29, 2026 Apify unified pricing across all four Clockworks TikTok actors. We got the email at 7:11 AM on June 30 and ran the audit before the day was done.

**Audit scores (June 30, 2026):**
- TikTok Profile Scraper (clockworks/tiktok-profile-scraper): 84/100 · 4.92/5, 33 reviews, 4.9M runs · winner
- TikTok Scraper (clockworks/tiktok-scraper): 79/100 · 4.69/5, 227 reviews, 71.5M runs · most battle-tested
- TikTok Data Extractor (clockworks/free-tiktok-scraper): 73/100 · 4.70/5, 57 reviews, 20M runs
- TikTok Hashtag Scraper (clockworks/tiktok-hashtag-scraper): 71/100 · 4.90/5, 24 reviews

**Price changes effective June 29, 2026, per 1,000 results:**
- TikTok Data Extractor: $5.00 to $3.00, 40% cheaper
- TikTok Profile Scraper: $4.00 to $3.00, 25% cheaper
- TikTok Hashtag Scraper: $5.00 to $3.00, 40% cheaper
- TikTok Video Scraper: $12.50 to $3.00, 76% cheaper, the biggest story

**Key finding:** price parity does not mean quality parity. The actors are not interchangeable, and the audit took 15 minutes.

**Use-case recommendations:**
- Influencer research: TikTok Profile Scraper, the only one with follower and following lists in the same run
- High-volume video or hashtag work: TikTok Scraper, widest input surface and most battle-tested
- Simple profile pulls: TikTok Data Extractor, lean and easy to configure
- Transcript and subtitle analysis: TikTok Video Scraper, viable for the first time after the 76% cut

**Answers to top questions:**
Q: Are Clockworks TikTok actors interchangeable after the price unification?
A: No. Despite identical $3.00/1K pricing, reliability, input surface and output completeness differ significantly. Profile Scraper scored 84 and Hashtag Scraper scored 71, which is 13 points apart at the same price.

Q: Which TikTok actor scored highest?
A: TikTok Profile Scraper (clockworks/tiktok-profile-scraper) at 84/100, driven by a 4.92/5 rating, the lowest 30-day failure rate, and follower and following scraping added in June 2026.

Q: Is TikTok Video Scraper worth it now?
A: Yes. The 76% price drop from $12.50 to $3.00 makes it viable for the first time, and it is the only actor in the suite with subtitle and transcript support.

---

## Frequently Asked Questions

**What does Audit Tools do?**
Evaluates Apify actors empirically and tells you which one to use for your task. You paste actor IDs, choose criteria, and get evidence-backed scores with the receipts.

**Who is Audit Tools for?**
Developers building pipelines, AI agents selecting tools autonomously, teams comparing actors before committing spend, and anyone burned by an actor that looked fine in the Store and failed on real inputs.

**Is Audit Tools affiliated with Apify?**
No. Independent evaluation layer. Reads public Apify Store data and runs actors through the standard Apify API.

**How many actors does it cover?**
The full public Apify Store catalog, about 41,000 actors, counted live on the site. Not a curated subset.

**What is free?**
The metadata and estimate modes, with no account and no card. Free criteria are rot check, 30-day reliability, listing freshness, pricing transparency, publisher pulse, x402 payability, permission posture, user trajectory, schema validity, machine-readable contract, adoption momentum, input schema completeness and documentation, README length and examples, run history depth, and quality delta.

**What costs money?**
The five criteria that require the actor to actually run: advertised-field truth, cost per usable item, end-to-end latency, run-to-run consistency, and memory fit. They come with the deep scan. README structure needs no actor run but is scored by an LLM judge, so it is metered separately.

**How much is the deep scan?**
$1.00 flat per evaluation over x402, in USDC on Base, one actor per request. Buying it with a card on audit-tools.ai costs the same $1.00, sold in blocks of five or more on a first purchase.

**Can an AI agent call Audit Tools directly?**
Yes. It is x402-native and listed in Coinbase's public x402 directory. Agents pay per deep scan in USDC on Base. No Apify account, no OAuth, no human in the loop. POST /api/evaluations with targets and mode.

**What is x402?**
A payment protocol for autonomous agents. Agents pay per API call in USDC on Base with no human approval step.

**Does Audit Tools run the actors?**
Yes, for the funded criteria. The actor executes on real inputs and Audit Tools inspects the output. The free tier reads public platform stats and static signals without executing actors.

**How is Audit Tools different from Store search?**
Search ranks by keyword and popularity. Audit Tools evaluates empirically: live probes, output inspection, true cost per usable result. Discovery against evaluation.

**How is Audit Tools different from a curated recommender list?**
A curated list can only rank the actors someone already shortlisted, using the stats the Store hands out. Audit Tools works across the full catalog, probes actors live, and reports the evidence behind each score.

**How is Audit Tools different from Apify's quality score?**
Apify's quality score is publisher-facing and helps a developer improve their Store ranking. Audit Tools is buyer-facing and helps you pick the right actor. Different job, different seat at the table.

**How do I know the scores are not just opinion?**
Every score carries the evidence it came from: run counts, dates, pricing model, permission flags, or inspected output. A criterion that cannot be measured is reported as unmeasured instead of guessed.

**What are custom criteria?**
Your own standards, written in plain language, scored 0 to 100 by an LLM judge against actor metadata and README with evidence. For example "handles rate limiting gracefully and returns typed output". They sit alongside the 23 built-ins and roll into one score.

---

## Agent usage pattern

```
1. Agent has a scraping task, e.g. "scrape TikTok profiles for influencer research"
2. Agent calls: POST https://audit-tools.ai/api/evaluations
   with targets = candidate actor IDs, mode = "metadata" (free)
3. Audit Tools returns scored results with evidence
4. Agent selects the highest overall score, or filters on the criteria that matter
5. Agent runs the selected actor via the Apify API
6. Agent repeats step 2 before each task, so it always uses the best actor available today
```

x402 payment flow for the paid deep scan:
- Agent receives 402 Payment Required with the x402 payment details
- Agent pays $1.00 in USDC on Base, with no human approval
- The request proceeds automatically

---

## Static pages (no JavaScript required)

- https://audit-tools.ai/faq.html - full FAQ with platform-specific questions
- https://audit-tools.ai/methodology.html - probe types, evidence classes, how scores are produced
- https://audit-tools.ai/blog/youtube-transcript-actors-july-2026.html - eight YouTube transcript actors audited, July 2026
- https://audit-tools.ai/blog/ai-agent-bought-an-audit.html - an agent bought a report over x402, July 2026
- https://audit-tools.ai/blog/tiktok-actors-june-2026.html - TikTok actor audit, June 2026
- https://audit-tools.ai/openapi.json - machine-readable API spec
- https://audit-tools.ai/llms.txt - LLM context, summary
- https://audit-tools.ai/llms-full.txt - LLM context, full (this file)
- https://audit-tools.ai/sitemap.xml - all URLs

---

Audit Tools · https://audit-tools.ai · Not affiliated with Apify