Methodology
How MediaReceipts selects claims, assigns verdicts, and maintains the quality and fairness of its record. Every element of this process is public and auditable.
Two real claims, one from Fox News, one from CNN, walk through every step of the pipeline, from broadcast to published verdict. Same methodology, same standard, applied identically.
Read the full walkthrough →What counts as a claim
Not everything said on air is a factual claim. MediaReceipts tracks only verifiable factual assertions: statements that make a specific, testable claim about the world that can be confirmed or contradicted using authoritative sources.
A valid claim has three elements: an identifiable subject, a statement about a concrete fact, and enough specificity that a fact-checker knows what to look for. The question we ask is: could a trained researcher determine whether this statement is true or false by consulting primary sources? If yes, it's in scope.
- Specific statistics and figures ("unemployment fell to 3.4%")
- Claims about events ("the Senate voted 62–38")
- Official findings and rulings ("the court ruled X unconstitutional")
- Attributions ("the President said…", when the quote is checkable)
- Claims about capabilities or institutional facts
- Polling and survey data citations
- Opinions, predictions, and value judgments ("this is a disaster")
- Editorial characterizations without testable predicates
- Rhetorical questions and non-declarative statements
- Claims requiring non-public information to verify
- Vague assertions without a falsifiable referent ("things are going well")
- Personal conduct and private life: only professional, on-air statements
How claims are selected
Claims currently enter the MediaReceipts pipeline through editorial selection by the MediaReceipts team, identifying factual claims from monitored cable news programming on tracked figures. A public nomination channel is in development; when launched, community-submitted clips and claims will enter the same review pipeline as editorial-sourced claims and not every nomination will lead to a published verdict.
Editorial selection maintains rough parity across all tracked figures and networks when selecting which claims to prioritize. Because nomination volume may skew toward figures from one end of the political spectrum, MediaReceipts actively monitors nomination patterns and manually balances the workload. Aggregated nomination data by network is published publicly so anyone can verify that coverage is balanced.
Source transcripts are obtained from broadcast audio of the tracked program. The primary method is machine transcription of the broadcast itself, producing a speaker-aligned, timestamped transcript. Where machine transcription is unavailable, the Internet Archive TV News Archive is the configured secondary source. No paywalled or private transcript services are used. Each transcript records how it was obtained, and that provenance is exposed on a claim's public record via the episode_source field.
Source standards
MediaReceipts uses a six-tier source hierarchy. Verdicts require sources from the appropriate tier for the claim type. Primary sources (Tiers 1–3) are sufficient on their own to support a verdict. Secondary sources (Tiers 4–6) require independent corroboration from at least two sources.
Source independence matters as much as source tier. Five outlets reporting the same government press briefing represent one source, not five. When all confirming sources trace to a single institutional origin, this is flagged in the decision memo and affects the confidence level assigned to the verdict.
| Tier | Category | Examples |
|---|---|---|
| 1 | Primary Institutional Records | Congressional Record, BLS/BEA/Census data releases, court filings, legislation text, official vote tallies, executive orders |
| 2 | Peer-Reviewed Research & Official Statistical Releases | Peer-reviewed journals, WHO/OECD/IMF data, CBO/GAO analyses, National Academy reports |
| 3 | Direct Official Statements with Traceable Provenance | Press conference transcripts, C-SPAN footage, verified official social media posts, on-record attributed quotes in press pool reports |
| 4 | Major Wire Services & Papers of Record | AP, Reuters, AFP dispatches; New York Times, Wall Street Journal, Washington Post, BBC, NPR, when directly citing primary documents |
| 5 | Subject-Matter Expert Analysis | Named, credentialed experts at recognized institutions commenting within their domain of expertise: economists, legal scholars, military analysts |
| 6 | Reputable Secondary Reporting | Established outlet investigative reporting, recognized fact-check organizations (PolitiFact, FactCheck.org), named-source reporting in established trade publications |
The two-gate review process
Every claim passes through two sequential editorial gates before a verdict is published. This structure separates the initial viability check (is this claim worth fact-checking?) from the substantive research and verdict assignment.
The human editor is always the quality gate. AI tools assist with research at scale, but no verdict is published without a human editor's review and sign-off. The AI's role is to surface evidence and flag patterns, not to determine outcomes.
Gate 1 selection criteria
Gate 1 is the platform's viability triage. It does not assign a verdict. It answers a single question for each extracted claim: is this claim worth the cost of deep verification at Gate 2? The criteria below govern that decision.
The four assessment dimensions
Every claim entering Gate 1 is assessed across four dimensions in sequence. Each dimension feeds the next, and the four together produce a single binary nomination: approve forward, or reject with a coded reason.
| Dimension | Approve signal | Reject / reduced-confidence signal | Decision-maker |
|---|---|---|---|
01 Claim structure Is the extracted claim a discrete, falsifiable assertion? | Identifiable subject, testable predicate, concrete referent a record can confirm or refute. Rhetorical characterizations that assert something specific about observable reality (governmental structure, legal status, measurable outcome, physical event) route forward. They carry a verifiable factual core. | Vagueness, pure opinion, motive attribution, rhetorical questions, predictions. Hard stop: if the extracted text does not faithfully represent what the speaker said, reject as EXTRACTION_ERROR. The AI does not proceed on a structurally invalid record. | AI nominates |
02 Attribution quality Can the claim be cleanly tied back to the speaker? | Machine transcription of the actual broadcast paired with a verbatim quote from a named speaker. | Journalist's reconstruction in place of direct broadcast capture (mediation layer recorded, attribution confidence reduced, but proceeds). Hedged or anonymous attributions in the source language ("officials say," "sources familiar with") are flagged but do not by themselves trigger rejection. | AI nominates |
03 Domain-based source availability Is the domain one where authoritative, independent sources realistically exist? | High availability: domestic policy, legislation, public economic and health statistics, official findings. Medium-high (with scrutiny): polling and survey data, with extra scrutiny on framing and temporal selection. | Low availability: active military operations, classified programs, sealed legal proceedings, with verification confidence capped. Very low: diplomatic negotiations, intelligence assessments, typically rejected unless they reference a publicly released document. | AI nominates |
04 Verification output interpretation What did the preliminary research pass actually find? | True with independent sources (strongest signal). True with single-origin sourcing (approved with sourcing flag). Mostly True (approved; specific inaccuracy documented). Misleading (approved; among the most editorially valuable findings; trigger and distortion mechanism documented). False (approved; core to the platform's mission). | Unverifiable in a structurally constrained domain: reject. On a recent claim that may not yet be indexed, the call goes to human judgment. | AI nominates |
Two-tier rejection codes
Every Gate 1 rejection carries a coded reason. Codes are split into two tiers. Tier 1 codes can be assigned by either the AI verification pass or a human reviewer. Tier 2 codes are reserved for human reviewers; they capture editorial judgments the AI is not authorized to make.
Why the two-tier split exists. It prevents the AI from both approving and rejecting the same fact pattern. When sources are single-origin, the AI flags the limitation in its reviewer summary and nominates approve. The human reviewer then decides whether the sourcing gap is disqualifying and, if so, applies SINGLE_SOURCE_INSTITUTIONAL as an override.
Reviewer actions
A Gate 1 reviewer has three actions on any claim card.
What rejection at Gate 1 means
Rejection at Gate 1 removes a claim from the verification pipeline. It does not delete the claim. Every rejected claim is retained in the platform's database with its rejection reason, the reviewer who made the call, the time of rejection, and (where applicable) any senior-reviewer disposition notes from a prior flag. This record is internal. Rejected claims do not appear on figure profiles, network scorecards, or platform-wide accuracy figures, because they have not been verified and have not received a verdict.
The rejection record is permanent and traceable. If a rejection is challenged (by the figure named, by an outside researcher, or by a contributor), the platform can produce the full chain of reasoning that produced it.
Quality controls
Consistent verdicts across different editors and different claims require structured quality controls. The standards below are drawn from academic content analysis. Until a second human reviewer is in place, the founding editor is the sole human reviewer; the second-review sampling and inter-rater reliability mechanisms described below will activate once second-reviewer positions are filled.
When new editors are added, they will be required to score a minimum Cohen's κ of 0.7 on a set of 20 pre-scored calibration claims before being permitted to review claims independently. The calibration set covers all four verdict categories and includes claims from across the political spectrum.
Gate 2 verdict categories
MediaReceipts uses four verdict categories. Each has a precise, operationalizable definition designed to produce consistent results when applied to the same claim. The categories are ordered by the nature of the discrepancy, not by severity of implied wrongdoing.
A fifth outcome, Unverifiable, is used when a claim is a legitimate factual assertion but authoritative public sources sufficient to assess it do not exist. Unverifiable claims are retained in the internal database for future re-evaluation and do not count toward a figure's accuracy record.
The Misleading category
The Misleading verdict is the most carefully defined category in our methodology, because technically true statements can cause real informational harm. We apply Misleading only when the distortion is material, meaning a viewer who heard the claim would form a substantially different understanding of the issue than one who received the complete picture.
Every Misleading verdict must identify which of four defined triggers applies and provide specific sourced evidence for how the distortion operates.
The Misleading and False categories are currently collected internally and will be released publicly upon seating of the independent advisory board. The True and Mostly True categories are displayed publicly now. When Misleading verdicts are released, they will appear alongside a standard disclosure explaining that the verdict assesses informational effect, not speaker intent. See Section 12 for the full intent policy.
How accuracy figures are calculated
Every accuracy percentage shown on a figure profile, network scorecard, or platform-wide chart is a direct count of published verdicts. The platform does not apply a weighted formula, a confidence multiplier, a time-decay function, or a correction modifier to the displayed numbers. The number you see is the number of claims in each verdict category, expressed as a percentage of that figure's total published verdicts.
A figure's accuracy rate is the count of True verdicts divided by the count of all published verdicts on that figure. While the platform is in its True/Mostly True public verdict phase, the calculation includes only those two categories. When Misleading and False are released publicly, those categories will enter the calculation under the same direct-count method: Misleading and False verdicts will count toward "all published verdicts" but not toward "True verdicts." Network and platform-level accuracy figures aggregate the same way.
The platform intends to surface correction data alongside verdict records so that post-publication remediation by a figure or outlet is visible to the public. The display mechanism will be defined in a separate design specification. The original verdict's classification does not change regardless of correction status, consistent with the principle that a False claim was still False at the time it aired. During the True/Mostly True-only publication phase, percentage-based accuracy displays are suppressed; only verdict counts are shown publicly.
Corrections tracking
For every claim, the Gate 2 research pass includes a structured correction search at the time of verdict preparation. The Adjudicator checks the network's correction page, keyword-searches for correction language, scans the figure's on-air and social statements, and reviews external fact-checker feeds. Any correction found at research time is recorded with the verdict in five structured fields and is part of the verdict's published record.
Continuous post-publication monitoring, automated re-checking of published verdicts against new corrections as they appear in the wild, is in active development under a published design (Decision Memo DM-2026-010). When live, late-arriving corrections will be surfaced on the affected verdict, the figure's record, and a "last-checked" timestamp will be visible on every verdict so the act of re-checking is auditable, not just the result.
A correction does not change the original verdict. A False claim that was later corrected is still False at the time it aired. Corrections are recorded with the affected verdict and remain part of that verdict's permanent published record. How corrections are reflected in figure-level accuracy figures is described in §9.
Appeals process
Any individual (including the figure in question, their representatives, or members of the public) may submit a formal appeal of any published verdict. Appeals are reviewed on the merits. All verdict revisions are documented in a public changelog; original verdicts remain visible with a "revised" notation.
A note on intent
MediaReceipts verdicts assess the informational effect of a claim on a reasonable viewer, not the speaker's intent, motivation, or character. This applies to all four verdict categories, including Misleading.
The test for every verdict is purely effect-based: does the claim's specific factual content match the authoritative record? Does its construction, framing, or omissions create a false impression? These questions have answers grounded in evidence. Whether a speaker intended to mislead does not.
A Misleading verdict means: this claim, as stated, creates a false impression in a reasonable viewer's mind. It does not mean the speaker lied, intended to deceive, or acted in bad faith. Intent is unknowable from a transcript, and a methodology that required proving intent would be both editorially subjective and legally vulnerable.
This principle is stated here in full and will appear as a standard disclosure on every published Misleading verdict when that category is released publicly.
The trigger labels in Section 8 describe properties of the published claim's content, not the speaker's selection process or motive. Whether a speaker chose to omit, cherry-pick, frame, or rely on outdated material is unknowable from a transcript and outside the scope of any verdict.