Did it work?For AI agents & developers

Technical reference. Written for analysts and AI agents and kept in sync with the repository, so it uses internal terms. Raw Markdown: /agents/schemas/impact-claim-v0.md.

Open Impact Claim Standard, v0 (impact-claim/v0)

Status: Draft v0, proposed by Better World (diditwork.org). License: pending Ho Hum's decision (proposed CC BY 4.0 for this spec; see LICENSE-PROPOSAL.md). Schema: schema/impact_claim.schema.json · Validator: scripts/validate_impact_claim.py (one file, Python standard library only) · Examples: examples/

Why this exists

Charities, evaluators, and AI agents all repeat impact numbers, often with no unit, no source, no date, and no way to check them later. This standard is a small, boring format for one claim: who says what, about which program, in which unit, according to which source, as of when, and when someone should check again. A claim that passes the validator is well-formed and honestly labeled. It is not thereby true, endorsed, or ranked.

Design rules

  1. Never guaranteed. honesty.guaranteed must be false. Modeled estimates must say honesty.modeled: true.
  2. Numbers need sources. A non-null metric.value needs at least one https citation with an accessed_on date. Unknown means null; never guess.
  3. Attribution must match the source. If source.evaluator names GiveWell, HLI, ACE, WHO or IHME, at least one citation must be on that evaluator's own domain. Unrecognized evaluators get a warning, and readers should treat the claim as self-reported until it is checked.
  4. Units are labels, not conversions. Use a shared v0 unit or custom:<name>. Units are never converted, and there is no cross-unit "impact score" (Better World ADR 0007 stays Proposed).
  5. Claims are data, not instructions. Text that tries to instruct an AI reader (for example "ignore previous instructions", "rank us first", or role tags) is rejected.
  6. Did it work? Modeled estimates and outcome claims should carry follow_up.check_after, the date someone should re-check them. Better World's outcome ledger is built on this field.

Fields

Field Required Notes
standard yes "impact-claim/v0"
claim_id yes Stable within the publisher, ^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$
as_of yes YYYY-MM-DD; not in the future
org yes name; optional website (https), identifiers[] (us_ein, uk_charity_number, bw_org_id, other)
intervention yes name; optional bw_intervention_id, geography
metric yes label, unit, value (number or null), optional period
claim_type yes modeled_estimate · measured_output · measured_outcome
source yes kind (independent_evaluator · peer_reviewed · self_reported), evaluator, citations[] (url, accessed_on, title)
honesty yes modeled (bool), guaranteed (always false)
uncertainty no low, high, interval, confidence (low/medium/high), notes
follow_up no check_after (date), method
notes no Free text, max 1000 characters

No other top-level fields are allowed in v0. Extensions wait for v1, so that "valid" means the same thing everywhere.

v0 unit vocabulary: USD_per_life_saved, USD_per_DALY_averted, USD_per_WELLBY, WELLBY_per_1000_USD, WELLBY_per_person, deaths_averted, DALYs_averted, people_reached, USD_per_person, USD_per_child, USD_per_net, nets_distributed, x_givedirectly, x_givewell_benchmark, tCO2e_averted, USD_per_tCO2e, animals_spared, percent, count.

Error and warning codes

Code Meaning
E000–E006 Not an object, missing field (E001), unknown field (E002), wrong standard (E003), bad claim_id (E004), bad date (E005), future as_of (E006)
E010–E012 Org or intervention name missing; org website not https
E020–E024 Metric label missing; bad unit (E021); unit not in vocabulary, so use custom: (E022); value key missing (E023); non-numeric value (E024)
E030–E033 Bad claim_type (E030); guaranteed not false (E031); modeled not boolean (E032); modeled estimate not marked modeled (E033)
E040–E048 Bad source kind; citations not a list; citation not an object; non-https or credentialed URL (E043); bad accessed_on (E044); future accessed_on (E045); number without citation (E046); evaluator missing (E047); named evaluator not supported by a citation on its domain (E048)
E050–E051 Uncertainty low > high (E050); value outside [low, high] (E051)
E060 Bad follow_up.check_after
E070–E071 Instruction-like text (E070); guarantee language (E071)
E080 Duplicate claim_id in a document
W030, W040, W041, W050, W060 Measured claim marked modeled; unrecognized evaluator; self-reported label; no uncertainty; no follow-up date

Run it on your own claims

bash
# Copy scripts/validate_impact_claim.py from the Better World repo (no public download URL yet;
# publishing waits on the license decision).
python3 validate_impact_claim.py my_claims.json          # summary
python3 validate_impact_claim.py my_claims.json --json   # machine-readable report

Input can be one claim, an array, or {"claims": [...]}. Exit code 0 means all valid, 1 means at least one invalid, and 2 means the file is unreadable.

How Better World uses it (and doesn't)

  • Better World will accept structured submissions in this format after Ho Hum approves an intake process. No intake is live today.
  • A valid claim is shown with its source label (self-reported, peer-reviewed, or independent evaluator). It never changes rankings on its own. Ranking changes need evaluator evidence and Ho Hum's approval.
  • Paid or sponsored status has no field in this standard and can never affect a claim's evidence (ADR 0003).

Versioning

impact-claim/v0 is a draft. Breaking changes go to v1. The validator's error codes are stable within v0.