Methodology
How MediaReceipts selects claims, assigns verdicts, and maintains the quality and fairness of its record. Every element of this process is public and auditable.
Two real claims, one from Fox News, one from CNN, walk through every step of the pipeline, from broadcast to published verdict. Same methodology, same standard, applied identically.
Read the full walkthrough →What counts as a claim
Not everything said on air is a factual claim. MediaReceipts tracks only verifiable factual assertions: statements that make a specific, testable claim about the world that can be confirmed or contradicted using authoritative sources.
A valid claim has three elements: an identifiable subject, a statement about a concrete fact, and enough specificity that a fact-checker knows what to look for. The question we ask is: could a trained researcher determine whether this statement is true or false by consulting primary sources? If yes, it's in scope.
- Specific statistics and figures ("unemployment fell to 3.4%")
- Claims about events ("the Senate voted 62–38")
- Official findings and rulings ("the court ruled X unconstitutional")
- Attributions ("the President said…", when the quote is checkable)
- Claims about capabilities or institutional facts
- Polling and survey data citations
- Opinions, predictions, and value judgments ("this is a disaster")
- Editorial characterizations without testable predicates
- Rhetorical questions and non-declarative statements
- Claims requiring non-public information to verify
- Vague assertions without a falsifiable referent ("things are going well")
- Personal conduct and private life: only professional, on-air statements
How claims are selected
Claims currently enter the MediaReceipts pipeline through editorial selection by the MediaReceipts team, identifying factual claims from monitored cable news programming on tracked figures. A public nomination channel is in development; when launched, community-submitted clips and claims will enter the same review pipeline as editorial-sourced claims and not every nomination will lead to a published verdict.
Editorial selection maintains rough parity across all tracked figures and networks when selecting which claims to prioritize. Because community-sourced nomination volume can vary across figures and networks for many reasons, MediaReceipts actively monitors nomination patterns and balances the workload. Aggregated nomination data by network is published publicly so anyone can verify that coverage is balanced.
Source transcripts are obtained from broadcast audio of the tracked program. The primary method is machine transcription of the broadcast itself, producing a speaker-aligned, timestamped transcript. Where machine transcription is unavailable, the Internet Archive TV News Archive is the configured secondary source. No paywalled or private transcript services are used. Each transcript records how it was obtained, and that provenance is exposed on a claim's public record via the episode_source field.
Source standards
MediaReceipts uses a six-tier source hierarchy. Verdicts require sources from the appropriate tier for the claim type. Primary sources (Tiers 1–3) are sufficient on their own to support a verdict. Secondary sources (Tiers 4–6) require independent corroboration from at least two sources.
Source independence matters as much as source tier. Five outlets reporting the same government press briefing represent one source, not five. When all confirming sources trace to a single institutional origin, this is flagged in the decision memo and affects the confidence level assigned to the verdict.
| Tier | Category | Examples |
|---|---|---|
| 1 | Primary Institutional Records | Congressional Record, BLS/BEA/Census data releases, court filings, legislation text, official vote tallies, executive orders |
| 2 | Peer-Reviewed Research & Official Statistical Releases | Peer-reviewed journals, WHO/OECD/IMF data, CBO/GAO analyses, National Academy reports |
| 3 | Direct Official Statements with Traceable Provenance | Press conference transcripts, C-SPAN footage, verified official social media posts, on-record attributed quotes in press pool reports |
| 4 | Major wire services, papers of record, and established news organizations | Reporting by organizations that meet all three criteria: (1) published editorial standards with documented correction policies, (2) named reporters under editorial review, and (3) direct primary-source referencing. Defined by function and standards, not by medium. Opinion programming excluded. Examples: AP, Reuters, AFP, Bloomberg News; NYT, WSJ, WaPo, FT, The Economist; CNN, CBS News, NBC News, ABC News, CNBC, Bloomberg TV (news only); BBC, NPR, PBS |
| 5 | Subject-Matter Expert Analysis | Named, credentialed experts at recognized institutions commenting within their domain of expertise: economists, legal scholars, military analysts |
| 6 | Reputable Secondary Reporting | Established outlet investigative reporting, recognized fact-check organizations (PolitiFact, FactCheck.org), named-source reporting in established trade publications |
Each source cited in a verdict carries a classification of its informational independence: whether the source contributes information that does not trace to another source in the set. The classification distinguishes new information from repeated information, regardless of outlet diversity. The methodology recognizes four classifications.
Independent. The source contributes information that does not trace to another source in the set. Two releases from the same institution contributing different information are both Independent.
Same origin. The source's information on the relevant factual predicates traces to the same data origin as another source in the set. Outlets reporting the same press release or the same anonymous briefing are all Same origin regardless of outlet diversity.
Primary origin. The primary institutional record from which Same-origin sources derive. One source per claim maximum. This classification is permitted only when the overall source set on the claim traces to a single institutional origin. On claims with multiple independent sources, no single origin point exists, and Primary origin does not apply.
Unknown origin. The information origin cannot be determined from available evidence.
The classifying dimension is informational origin, not institutional provenance.
The two-gate review process
Every claim passes through two sequential editorial gates before a verdict is published. This structure separates the initial viability check (is this claim worth fact-checking?) from the substantive research and verdict assignment.
The human editor is always the quality gate. AI tools assist with research at scale, but no verdict is published without a human editor's review and sign-off. The AI's role is to surface evidence and flag patterns, not to determine outcomes.
Gate 1 selection criteria
Gate 1 is the platform's viability triage. It does not assign a verdict. It answers a single question for each extracted claim: is this claim worth the cost of deep verification at Gate 2? The criteria below govern that decision.
The four assessment dimensions
Every claim entering Gate 1 is assessed across four dimensions in sequence. Each dimension feeds the next, and the four together produce a single binary nomination: approve forward, or reject with a coded reason.
| Dimension | Approve signal | Reject / reduced-confidence signal | Decision-maker |
|---|---|---|---|
01 Claim structure Is the extracted claim a discrete, falsifiable assertion? | Identifiable subject, testable predicate, concrete referent a record can confirm or refute. Rhetorical characterizations that assert something specific about observable reality (governmental structure, legal status, measurable outcome, physical event) route forward. They carry a verifiable factual core. | Vagueness, pure opinion, motive attribution, rhetorical questions, predictions. Hard stop: if the extracted text does not faithfully represent what the speaker said, reject as EXTRACTION_ERROR. The AI does not proceed on a structurally invalid record. | AI nominates |
02 Attribution quality Can the claim be cleanly tied back to the speaker? | Machine transcription of the actual broadcast paired with a verbatim quote from a named speaker. | Journalist's reconstruction in place of direct broadcast capture (mediation layer recorded, attribution confidence reduced, but proceeds). Hedged or anonymous attributions in the source language ("officials say," "sources familiar with") are flagged but do not by themselves trigger rejection. | AI nominates |
03 Domain-based source availability Is the domain one where authoritative, independent sources realistically exist? | High availability: domestic policy, legislation, public economic and health statistics, official findings. Medium-high (with scrutiny): polling and survey data, with extra scrutiny on framing and temporal selection. | Low availability: active military operations, classified programs, sealed legal proceedings, with verification confidence capped. Very low: diplomatic negotiations, intelligence assessments, typically rejected unless they reference a publicly released document. | AI nominates |
04 Verification output interpretation What did the preliminary research pass actually find? | True with independent sources (strongest signal). True with single-origin sourcing (approved with sourcing flag). Mostly True (approved; specific inaccuracy documented). Misleading (approved; among the most editorially valuable findings; trigger and distortion mechanism documented). False (approved; core to the platform's mission). | Unverifiable in a structurally constrained domain: reject. On a recent claim that may not yet be indexed, the call goes to human judgment. | AI nominates |
Two-tier rejection codes
Every Gate 1 rejection carries a coded reason. Codes are split into two tiers. Tier 1 codes can be assigned by either the AI verification pass or a human reviewer. Tier 2 codes are reserved for human reviewers; they capture editorial judgments the AI is not authorized to make.
Why the two-tier split exists. It prevents the AI from both approving and rejecting the same fact pattern. When sources are single-origin, the AI flags the limitation in its reviewer summary and nominates approve. The human reviewer then decides whether the sourcing gap is disqualifying and, if so, applies SINGLE_SOURCE_INSTITUTIONAL as an override.
Reviewer actions
A Gate 1 reviewer has three actions on any claim card.
What rejection at Gate 1 means
Rejection at Gate 1 removes a claim from the verification pipeline. It does not delete the claim. Every rejected claim is retained in the platform's database with its rejection reason, the reviewer who made the call, the time of rejection, and (where applicable) any senior-reviewer disposition notes from a prior flag. This record is internal. Rejected claims do not appear on figure profiles, network scorecards, or platform-wide accuracy figures, because they have not been verified and have not received a verdict.
The rejection record is permanent and traceable. If a rejection is challenged (by the figure named, by an outside researcher, or by a contributor), the platform can produce the full chain of reasoning that produced it.
Quality controls
Consistent verdicts across different editors and different claims require structured quality controls. The standards below are drawn from academic content analysis. Until a second human reviewer is in place, the founding editor is the sole human reviewer; the second-review sampling and inter-rater reliability mechanisms described below will activate once second-reviewer positions are filled.
When new editors are added, they will be required to score a minimum Cohen's κ of 0.7 on a set of 20 pre-scored calibration claims before being permitted to review claims independently. The calibration set covers all four verdict categories and includes claims from across the political spectrum.
Gate 2 verdict categories
MediaReceipts uses four verdict categories. Each has a precise, operationalizable definition designed to produce consistent results when applied to the same claim. The categories are ordered by the nature of the discrepancy, not by severity of implied wrongdoing.
A fifth outcome, Unverifiable, is used when a claim is a legitimate factual assertion but authoritative public sources sufficient to assess it do not exist. Unverifiable claims are retained in the internal database for future re-evaluation and do not count toward a figure's accuracy record.
The methodology recognizes five specific patterns where a claim contains an imprecision that does not rise to the level of a factual error a competent fact-checker would note. These claims remain TRUE rather than crossing into MOSTLY TRUE.
Hedged approximation. When a speaker uses a verbal hedge such as “like 65, 66 percent” or “around 40, 45 percent,” the language is a hedge rather than a factual claim of precision. If the confirmed figure falls within the stated range, the imprecision is immaterial. If the confirmed figure falls outside the range, the hedge does not insulate the claim.
Specific outcome claims. Unqualified outcome assertions such as “nobody was hurt” or “no one was injured” are evaluated on their plain meaning. When the primary institutional record specifically documents an outcome different from the unqualified claim, the claim is MOSTLY TRUE. A contextual narrowing argument that the speaker probably meant a narrower category does not transform a record-contradicted outcome claim into an immaterial imprecision.
Statutory and technical-language compression. When a speaker describes a legal instrument, regulatory action, or technical specification by its common-usage name while omitting statutory elements or qualifying clauses, the description is treated as common-usage simplification rather than factual error if the instrument is correctly identified and the omitted elements do not assert anything false about it.
Directional understatement. When a claim describes a confirmed event less severely than the source material documents, the imprecision is immaterial provided every stated fact is confirmed and the understatement does not mischaracterize the nature of the event. Overstatement of severity may still qualify as an identifiable inaccuracy if the degree characterization is demonstrably wrong.
Live-broadcast phonetic proximity. When a broadcaster mispronounces a proper noun (a personal name, a facility name, a place name) at broadcast speed, the mispronunciation is treated as immaterial imprecision rather than identifiable inaccuracy when the spoken form is phonetically close to the correct name, every other contextual detail uniquely identifies the correct referent, and no reasonable viewer would identify a different person, place, or entity. If the contextual identification is ambiguous or the phonetic distance is large enough to denote a different referent, the mispronunciation is an identifiable inaccuracy and the claim is MOSTLY TRUE. This standard applies regardless of whether the mispronunciation originated with the speaker or with the audio-transcription layer.
The Misleading category
The Misleading verdict is the most carefully defined category in our methodology, because technically true statements can cause real informational harm. We apply Misleading only when the distortion is material, meaning a viewer who heard the claim would form a substantially different understanding of the issue than one who received the complete picture.
Every Misleading verdict must identify which of four defined triggers applies and provide specific sourced evidence for how the distortion operates.
The Misleading and False categories are currently collected internally and will be released publicly upon seating of the independent advisory board. The True and Mostly True categories are displayed publicly now. When Misleading verdicts are released, they will appear alongside a standard disclosure explaining that the verdict assesses informational effect, not speaker intent. See Section 12 for the full intent policy.
How accuracy figures are calculated
Every accuracy percentage shown on a figure profile, network scorecard, or platform-wide chart is a direct count of published verdicts. The platform does not apply a weighted formula, a confidence multiplier, a time-decay function, or a correction modifier to the displayed numbers. The number you see is the number of claims in each verdict category, expressed as a percentage of that figure's total published verdicts.
A figure's accuracy rate is the count of True and Mostly True verdicts divided by the count of all published verdicts on that figure. Both categories count in the figure's favor: a record made up entirely of True and Mostly True verdicts is a 100% accuracy rate. When Misleading and False are released publicly, those categories will enter the calculation under the same direct-count method. They will count toward total published verdicts but not toward the accurate side of the ledger, and they are the only categories that lower a figure's rate. Claims dispositioned Unverifiable are excluded from the calculation entirely, appearing in neither the numerator nor the denominator. Network-level and platform-level accuracy figures aggregate the same way.
The platform intends to surface correction data alongside verdict records so that post-publication remediation by a figure or outlet is visible to the public. The display mechanism will be defined in a separate design specification. The original verdict's classification does not change regardless of correction status, consistent with the principle that a False claim was still False at the time it aired. During the True/Mostly True-only publication phase, percentage-based accuracy displays are suppressed; only verdict counts are shown publicly.
Corrections tracking
For every claim, the Gate 2 research pass includes a structured correction search at the time of verdict preparation. The Adjudicator checks the network's correction page, keyword-searches for correction language, scans the figure's on-air and social statements, and reviews external fact-checker feeds. Any correction found at research time is recorded with the verdict in five structured fields and is part of the verdict's published record.
Continuous post-publication monitoring, automated re-checking of published verdicts against new corrections as they appear in the wild, is in active development under a published design (Decision Memo DM-2026-010). When live, late-arriving corrections will be surfaced on the affected verdict, the figure's record, and a "last-checked" timestamp will be visible on every verdict so the act of re-checking is auditable, not just the result.
A correction does not change the original verdict. A False claim that was later corrected is still False at the time it aired. Corrections are recorded with the affected verdict and remain part of that verdict's permanent published record. How corrections are reflected in figure-level accuracy figures is described in §9.
Appeals process
Any individual (including the figure in question, their representatives, or members of the public) may submit a formal appeal of any published verdict. Appeals are reviewed on the merits. All verdict revisions are documented in a public changelog; original verdicts remain visible with a "revised" notation.
A note on intent
MediaReceipts verdicts assess the informational effect of a claim on a reasonable viewer, not the speaker's intent, motivation, or character. This applies to all four verdict categories, including Misleading.
The test for every verdict is purely effect-based: does the claim's specific factual content match the authoritative record? Does its construction, framing, or omissions create a false impression? These questions have answers grounded in evidence. Whether a speaker intended to mislead does not.
A Misleading verdict means: this claim, as stated, creates a false impression in a reasonable viewer's mind. It does not mean the speaker lied, intended to deceive, or acted in bad faith. Intent is unknowable from a transcript, and a methodology that required proving intent would be both editorially subjective and legally vulnerable.
This principle is stated here in full and will appear as a standard disclosure on every published Misleading verdict when that category is released publicly.
The trigger labels in Section 8 describe properties of the published claim's content, not the speaker's selection process or motive. Whether a speaker chose to omit, cherry-pick, frame, or rely on outdated material is unknowable from a transcript and outside the scope of any verdict.
Public verdict reasoning
The reasoning paragraph that appears on a published verdict card is a public-facing argument, written for a reader with no knowledge of the platform's internal processes. It explains what the claim asserts, what the sources show, and why the verdict was reached.
The public reasoning is derived from the internal reasoning produced during Gate 2 verification, through an editorial pass that an editor completes at publication time. The editorial pass removes pipeline-internal terminology, translates methodology references into plain language, and ensures the reasoning reads as a self-contained argument. The internal reasoning is preserved unmodified in the platform's records for audit.
Publication is gated on the public reasoning being present. A verdict cannot be published until the editorial pass has produced the public-facing version. The editorial-pass workflow is enforced by an automated check against a list of prohibited content patterns (internal check numbering, references to inter-stage verdict disagreements, schema field names, methodology-internal terms of art used without plain-language explanation). Drafts containing any of those patterns are returned for correction before publication.