1. Identify the claim
- Copy the exact claim.Do not silently strengthen “associated with” into “caused.”
- Name the claimant and data collector.Record whether the result is first-party, vendor-produced, customer-reported, or independently replicated.
- Classify the result.Paid appearance, organic mention, visible citation, retrieval inference, traffic, lead, sale, or a combined measure.
- Record the publication and measurement dates.AI-product behavior and interfaces change; a date-free result cannot be placed in context.
2. Define the test environment
- AI product and model/version are named.“AI search” is not a test environment.
- Interface and search state are recorded.Note whether visible web search, grounding, shopping, or advertising was active.
- Markets, languages, accounts, and device conditions are disclosed.These can change the available experience and result.
- The test window is bounded.Record exact dates rather than “recently” or “after optimization.”
3. Inspect the measurement
- The exact prompt set is available.Separate brand-free discovery, comparison, problem, and known-brand questions.
- Prompt selection occurred before results were seen.Post-result selection can turn a favorable subset into the headline.
- Run counts and denominators are published.A rate without the number of opportunities cannot be interpreted.
- Metric definitions and formulas are explicit.Terms such as visibility, share of voice, alignment, and citation rate need operational definitions.
- Raw counts accompany percentages and multiples.“Nine of 20” is more inspectable than “45%.”
4. Test the comparison
- The baseline is shown.A lift requires both the starting value and the later or comparison value.
- Compared observations are reasonably matched.Check prompts, dates, products, settings, markets, and run counts.
- Only one intended intervention changed.Multiple simultaneous changes weaken causal attribution.
- A no-change or control condition exists when feasible.This helps separate treatment movement from ordinary volatility.
- Variation and uncertainty are reported.Repeated runs should show stability, range, or another honest account of noise.
5. Check the reporting
- Paid, organic, and combined visibility are separated.A paid appearance is not evidence of organic citation authority.
- Null, negative, and excluded observations are disclosed.Readers need to know what did not work and what was removed.
- Limitations sit near the headline result.A caveat should travel with the claim when the page is quoted or extracted.
- The result is not generalized beyond its sample.A campaign result is not automatically a category or industry benchmark.
- Replication materials are accessible.Prompts, logs, page versions, and calculation notes should be inspectable where privacy permits.
6. Decide how to use the claim
| Evidence state | Responsible use |
|---|---|
| Method and raw observations are reproducible | Report the result with its sample, environment, dates, and limitations. |
| Method is partly disclosed | Attribute the result to the publisher and identify the missing evidence. |
| Only a headline metric is available | Treat it as a marketing claim, not a benchmark or planning assumption. |
| Paid and organic states are combined | Do not use it to support a claim about organic discovery or citation authority. |
| No denominator or baseline is available | Do not repeat the percentage or lift without qualification. |
Minimum reporting sentence: “[Publisher] reports [result] from [sample and environment] during [dates]. The public material does not disclose [material missing evidence], so the result has not been independently reproduced.”
Make your own evidence easier to inspect.
Start by auditing the page that states the claim. Clear entities, visible sources, defined terms, and disclosed limitations make evidence more useful to readers and machines.
Audit a page free