AI answer auditing and correction checklist

What this checklist covers

An AI answer audit turns a suspected error into a documented case that another person can review. The work begins by preserving the prompt, answer, citations, product, conditions, and date, then determines what is wrong, who may be affected, and which source or reporting path can address it.

Correction can mean updating a page, structured data, feed, profile, or internal record the organization controls. It can also mean sending evidence to another publisher or using an answer product’s feedback, policy, privacy, intellectual-property, or legal process. These actions have different purposes, and none guarantees that a future answer will change across every system.

Use this checklist to detect, capture, classify, investigate, correct, escalate, and recheck false, stale, incomplete, misleading, wrong-entity, citation-failure, or suspected-manipulation cases. Use unverifiable or not a correction case when that is the supported finding. The process reviews observed answers. The classification and triage playbook provides a decision tree and response workflow. The answer-capture log preserves the observed output, and the case-intake template turns a material problem into an owned investigation.

Read Auditing & Correcting first if you need the case model, evidence principles, response paths, closure standard, common failure patterns, or operating workflow before working through the individual checks.

Escalate urgent harm at intake

Before starting the numbered checks, assess whether the observed answer credibly presents urgent harm. If it does, preserve enough evidence to support the report and follow the appropriate urgent incident, policy, privacy, legal, safety, or specialist escalation path immediately. That response overrides the checklist’s normal order; do not wait to finish capture, classification, source tracing, or routine correction steps. Record the case and resume applicable investigation work when the urgent response permits.

On this page

Use the stage that matches the case. Do not delay a credible urgent escalation to complete every later-stage check.

• Intake and ownership: open an actionable, appropriately restricted case.
• Evidence capture: preserve observations before changes.
• Classification and severity: verify the claim and consequence.
• Source investigation: separate inspectable facts from inferred causes.
• Controlled corrections: fix and verify owning records.
• Outside requests: match the reporting path to the issue.
• Recheck and closure: record results and remaining risk.

Record a result for each applicable check

Use the check number in the case record. Mark pass, fail, pending, or not applicable with a reason; a checked box without evidence is not a verified finding. The triage playbook owns classification rules. The correction and recheck playbook owns execution and includes a completed hypothetical case.

Case ID / check number:
Applicable? Yes / no, with reason:
Result: pass / fail / pending / not applicable
Evidence location and observed result:
Missing evidence or required action:
Owner and due date:
Recheck result and remaining limitation:

Hypothetical result: HT-4 / check 46, schedule rechecks: pass. The case record sets September 17 for three saved runs under the original conditions and names the case owner. That passes the scheduling check; it does not establish that any answer has been corrected.

Establish intake and ownership

1. Create a clear reporting channel

Give employees, customers, partners, and reviewers a defined place to report a questionable answer. Ask for the product, prompt, date, answer, and available links without requiring the reporter to diagnose the cause.

2. Confirm that the reported output can be reviewed

Check that the evidence identifies the actual answer product and preserves enough of the interaction to understand the claim in context. Use the answer-capture log when the original output can still be obtained; do not begin a correction campaign from a paraphrase, cropped sentence, or recollection.

3. Assign a case identifier and owner

Create one case-intake record for the issue and assign the person responsible for triage, evidence, actions, and follow-up. Link duplicate reports to the same case while preserving differences in prompt, market, or output.

4. Define the affected entity and claim

State which organization, product, person, location, or other entity the answer describes and write the disputed claim plainly. This prevents a correction for one entity from being applied to another with a similar name.

5. Record the affected audience and decision

Identify who may see the answer and what decision it could influence, such as a purchase, visit, application, safety action, or reputation judgment. The likely consequence helps determine severity and response time.

6. Protect sensitive case information

Remove credentials, private conversations, personal data, and confidential business information that are not needed for the review. Restrict access when the evidence involves customers, employees, health, legal matters, security, or another sensitive subject.

7. Set status and response targets

Use clear states such as new, verifying, investigating, correcting, reported externally, monitoring, resolved, and closed. Set response and review targets based on severity rather than handling every mention as an emergency.

Capture the evidence

8. Save the exact prompt and conversation context

Preserve the user’s wording, relevant earlier turns, follow-up questions, and any selected mode or tool. A claim may depend on context that disappears when only the final question is retained.

9. Save the complete answer

Retain the full answer as text and preserve its visual presentation with a screenshot or export when possible. Include warnings, labels, tables, images, source panels, and surrounding language that changes how the disputed statement is understood.

10. Record the product and available version information

Save the product name, model or mode when displayed, app or browser surface, subscription tier, and whether web search or another tool was active. Do not infer an undisclosed model or retrieval process from the style of the response.

11. Record time and observation conditions

Keep the date, time, market, language, device, login state, personalization state when known, and any relevant location setting. These details support a comparable recheck and explain why another reviewer may receive a different output.

12. Preserve visible citations and links

Save the linked title, displayed publisher, destination URL, anchor or citation marker, and the claim each source appears to support. Capture redirects or broken links because the destination may change after the original observation.

13. Save source pages as they appeared

Retain a dated copy, screenshot, or archive reference for material source content when organizational policy and applicable rules permit it. A live page can change during the investigation, leaving the team unable to show what the answer or correction relied on.

14. Record reproducibility without overwriting the original

Repeat the prompt under documented conditions when the issue warrants it, but store each result as a separate observation. A different answer does not erase the original, and a repeated error does not prove that every user receives it.

Classify and assess the issue

Use the classification decision tree when a reviewer needs to distinguish issue type from consequence-based severity and choose the appropriate response path.

15. Classify a false statement

Use false when reliable current evidence directly contradicts a material factual claim without an established formerly correct state. If prior accuracy is established, use stale; if history is unknown, record that limit. Record the precise sentence, the corrected fact, and the evidence instead of labeling the entire answer false when only one part is wrong.

16. Classify stale information

Use stale when a statement was accurate in an earlier period but no longer reflects the current status, version, ownership, availability, price, location, or role. Preserve the effective date of the change and the former fact so the transition is clear.

17. Classify an incomplete answer

Use incomplete when a material omission changes the meaning or leaves the reader unable to make the intended decision. Do not treat every missing detail as an error; identify the consequence of the omitted condition or context.

18. Classify misleading context

Use misleading when individual facts may be technically correct but their framing, comparison, implication, or missing qualification creates a materially wrong impression. Explain the interpretation a reasonable reader could take and which context would correct it.

19. Classify entity confusion

Use wrong entity or ambiguous identity when the answer combines people, companies, products, or locations that should be distinct. Record the identifiers, canonical URLs, locations, relationships, or other facts that disambiguate them.

20. Classify citation and source failures

Distinguish a broken link, irrelevant citation, source that does not support the claim, misquoted source, and apparently fabricated reference. A citation marker can make an answer look supported even when the linked material says something different.

21. Classify impersonation or manipulated information

Record suspected fake profiles, counterfeit sites, deceptive redirects, fabricated documents, or coordinated material that may be influencing the information environment. Preserve evidence and involve security, legal, fraud, or platform-trust teams when the case exceeds an editorial correction.

22. Assess severity and urgency

Rate the likely harm, audience reach, decision consequence, evidence strength, persistence, and sensitivity of the subject. Give immediate attention to credible threats involving health, safety, finance, legal rights, security, fraud, discrimination, or imminent operational harm.

23. Separate disagreement from correctable error

Distinguish a verifiable factual problem from opinion, criticism, ranking, taste, or an unresolved expert dispute. A negative answer is not automatically inaccurate, and a correction process should not be used to suppress legitimate reporting or unfavorable views.

24. Choose the appropriate response level

Decide whether the case needs documentation only, a first-party correction, a publisher request, product feedback, a policy report, specialist escalation, or an urgent incident response. Use the least expansive action that addresses the actual issue while preserving escalation for higher-risk cases.

Trace the source and cause

25. Establish the correct fact

Identify the current authoritative evidence and the person qualified to approve the correction. Resolve internal disagreement before asking outside publishers or platforms to choose among conflicting claims from the same organization.

26. Open and evaluate every cited source

Check whether each cited page contains the disputed claim, whether the content is current, and whether the answer represented it accurately. Record direct support, contradiction, partial support, and uncertainty separately.

27. Search for likely uncited sources

Look for pages, profiles, feeds, cached records, and syndicated copies that contain the same wording or obsolete fact. Describe matching wording alone as a possible source. Use likely only with stated supporting evidence; reserve confirmed for a directly established connection, such as a product disclosure.

28. Audit first-party pages

Review the canonical page, About and contact information, product pages, author profiles, location pages, policies, and relevant editorial content. Find contradictions and ambiguous wording within the site before attributing the problem entirely to an outside system.

29. Audit structured data and feeds

Compare visible content with structured data, product or merchant feeds, APIs, catalogs, and other machine-readable records. Correct the system that owns the field so the same obsolete value is not republished after a display-layer edit.

30. Audit maintained public profiles

Check business listings, social profiles, marketplaces, app stores, directories, and other accounts the organization controls or is authorized to manage. Retire duplicates and update old descriptions through the platform’s legitimate ownership process.

31. Review relevant independent sources

Identify reliable third-party pages that repeat or contradict the disputed claim and determine whether they have a correction policy. Keep independent reporting separate from organization-supplied profiles, syndicated press material, and copied directory records.

32. Check technical delivery and freshness

Verify that the corrected page can be fetched and rendered, returns the intended status, uses the expected canonical URL, and is not blocked from the relevant crawler. Review caches, update signals, and indexing reports when an old version continues to appear.

33. Document the probable cause and uncertainty

Record whether the evidence points to an outdated source, conflicting first-party records, identity ambiguity, citation misuse, technical delivery, manipulated content, or an unknown cause. Keep alternative explanations open when the answer system does not reveal its complete source or generation process.

Correct information the organization controls

34. Fix the canonical first-party source

Update the page that should serve as the durable reference for the fact and make the correction explicit enough for a reader to understand. Do not create thin duplicate pages solely to repeat the preferred statement across more URLs.

35. Reconcile every controlled representation

Update structured data, feeds, profiles, documents, media kits, APIs, and related pages that repeat the incorrect value. Use the canonical fact record and field owner to prevent one system from restoring the old information later.

36. Clarify identity and relationships

Add the names, identifiers, URLs, locations, ownership, dates, or product relationships needed to distinguish the entity. Correct ambiguity without stuffing pages or markup with unrelated references.

37. Correct the editorial explanation

Revise wording that is technically accurate but unclear, unsupported, or easy to misinterpret. Add necessary evidence and qualifications near the claim, then have the responsible expert and editor approve the change.

38. Preserve a meaningful correction record

Record what changed, why it changed, who approved it, and when the new fact became effective. Publish a correction note when the earlier version materially affected a reader’s understanding rather than silently rewriting significant errors.

39. Address the process failure

Identify the workflow, owner, integration, template, or review gap that allowed the incorrect fact to spread or remain stale. Add a validation or update trigger so the same failure is less likely to recur.

Request outside corrections and report issues

40. Contact the source publisher when appropriate

Use the publisher’s correction process and provide the exact page, disputed wording, corrected fact, primary evidence, and a concise explanation. Request a factual review without demanding removal of legitimate opinion or independent coverage.

41. Use in-product answer feedback

Use the answer product’s current feedback control for inaccurate, unhelpful, biased, or otherwise problematic output when that channel fits the issue. Include the prompt, claim, evidence, and affected entity, and preserve the submitted report because the interface may not provide a case history.

42. Use policy, safety, privacy, or legal channels correctly

Use a dedicated reporting form when the issue involves prohibited content, threats, fraud, impersonation, personal data, intellectual property, or another defined policy or legal concern. Do not misclassify an ordinary factual dispute as a violation to obtain faster handling.

43. Claim or verify eligible profiles and panels

Use legitimate ownership or representative verification for business profiles, knowledge panels, and other managed records when available. Verification may allow suggestions or edits, but it does not give direct control over every fact or guarantee acceptance.

44. Escalate high-risk cases to qualified specialists

Involve legal counsel, security, privacy, safety, compliance, communications, or executive leadership according to the organization’s incident policy. Preserve evidence and avoid public speculation when an investigation, regulatory duty, or personal safety concern requires controlled handling.

45. Log every external action and response

Record the channel, submission date, evidence supplied, case number, response, stated status, and follow-up date. Separate an acknowledged report from an accepted correction and both from a changed public output.

Recheck, communicate, and close

46. Allow for source and system update time

Set a recheck date based on the source, indexing path, product, and severity rather than repeatedly resubmitting the same report. Use urgent escalation paths for urgent harm, but do not interpret ordinary processing delay as proof that a correction was rejected.

47. Recheck comparable conditions

Repeat the original prompt with the recorded market, language, mode, and account conditions where practical. Save the new output as another observation and note any provider or product changes that make the comparison imperfect.

48. Check the source layer separately from the answer

Confirm that first-party pages, markup, feeds, profiles, and relevant outside sources now show the intended information. A corrected source is a completed controlled action even when an answer has not changed, while a changed answer does not prove every source is correct.

49. Sample beyond one successful recheck

Review more than one run or related prompt when the issue is material and the product’s answers vary. Do not close a persistent-risk case solely because one later answer omitted the error.

50. Communicate status with precise language

Tell stakeholders whether the error was verified, the source was corrected, a report was submitted, a response was received, or later outputs changed. Avoid saying the issue is fixed everywhere when the evidence covers only one source, product, prompt, or observation.

51. Close with evidence and a reason

Close the case when the assigned actions are complete, the remaining risk is accepted by the appropriate owner, and follow-up evidence is attached. Reopen it when the same material claim recurs or new evidence changes the assessment.

52. Feed lessons back into prevention and measurement

Add recurring facts, prompts, sources, and failure patterns to the entity record, editorial review, technical checklist, and measurement plan. Report correction time, recurrence, unresolved severity, and controlled actions without treating provider response as an outcome the organization can guarantee.

Choose the first audit

Begin with one material answer and preserve it in the answer-capture log before changing any source. Open a case-intake record, verify the fact, assess the consequence, trace the supporting information, correct the records you control, choose the appropriate outside path, and schedule a comparable recheck with one accountable owner.