Trust Methodology

Ethos publishes two kinds of records: audits — structured, operator-attested evaluations of a deployed AI system — and community reports — first-hand or documented accounts of harm, bias, or failure. Both are produced, scored, and verified by the same trust process, so a reader can compare records with equal confidence in how they were made.

This page describes that process end to end. For the full scoring rubric and framework crosswalk, see the Registry Methodology.

1. Submission. An operator (or a reviewer on their behalf) submits a system profile at /audit/new with intended use, deployment context, and a verifiable public reference.
2. Rubric response. The submitter answers a 10-question NIST-aligned instrument covering the Govern (4 questions) and Map (6 questions) functions of the AI RMF.
3. Evidence attachment. Each "yes" or "partial" response must attach at least one artifact — policy, model card, DPIA, filing, or public commitment — or it is downgraded during review.
4. Intake validation. Automated checks confirm completeness, deduplicate against existing records, and flag obvious data-quality issues before a human is engaged.

A few real records from the registry, showing how questionnaire responses roll up into a Transparency Score. Each card lists the answers for every rubric question; expand the walk-through below to read the evidence and reviewer notes attached to the highest-scoring example.

Each response is scored Yes = 10, Partial = 5, or No = 0, producing a composite Transparency Score from 0 to 100. Governance contributes 40 points, Map contributes 60.

• Scores reflect disclosed governance, not runtime behavior. A high score means the operator has documented and attested to specific controls — verified against the artifacts they attached.
• The Community Signal — verified incidents and framework flags — is shown next to the score, but does not modify it. This separates what the operator claims from what the public has observed.
• Scores are re-computed automatically whenever any underlying response is amended through a correction or appeal.

1. First reviewer. A Community Reviewer confirms that each cited artifact exists at the referenced URL, matches the response category, and was published by the named operator. Conflicts of interest are declared before assignment.
2. Second reviewer. An independent reviewer re-scores the rubric from the artifacts alone, without seeing the first reviewer's scores. Discrepancies trigger a third editorial pass.
3. Editorial sign-off. The Registry Steward approves publication. Every reviewer identifier and the review timestamp are recorded on the public record.

Community reports come from three source paths:

• Direct submissions filed against a system card in the registry, with an evidence URL, bias category, and description.
• Curated feeds such as the AI Incident Database (AIID) and public regulatory filings.
• Signed submissions from researchers, journalists, and affected communities, submitted by email with supporting documentation.

Every accepted report must include an identifiable operator, a date or date range, a description of the harm or failure, and at least one verifiable source. Anonymous reports are accepted but held to a higher corroboration bar before publication.

1. Source check. The reviewer confirms the referenced artifact resolves, is authored or attributable to a credible party, and describes the events claimed.
2. Corroboration. Where possible, at least one independent source is required. Single-source reports are labeled as such on the public record.
3. Attribution check. The reviewer confirms the system and operator named in the report match a real, identifiable deployment — not a rumor or a mislabel.
4. PII removal. Identifying details of harmed individuals are removed unless the individual has consented in writing to be named.
5. Second-reviewer sign-off. A reviewer with no prior involvement confirms each of the above before the record is queued for publication.

Before any adverse record is published, the named operator is contacted at their published accountability address and given a 10 business-day window to respond. Operator responses are published alongside the record — not in place of it. If the operator declines to respond, that fact is also recorded.

Every published record shows: the underlying sources, the reviewer identifiers, the review and publication timestamps, the operator-notice status, and a permanent, citable URL.

Any operator or affected party may file an appeal within 30 days of publication. Appeals are handled by a reviewer with no prior involvement and resolved within 15 business days. Outcomes are one of Uphold, Correct, or Retract. The original text, the correction, and the rationale are shown together on the public record. Records are never silently deleted.

• Incident feeds refresh continuously.
• Audit records are re-reviewed at least annually or on material change disclosed by the operator.
• Reviewer rotation — no reviewer may be assigned to the same system twice in a row.
• This methodology is versioned; changes are published with a dated changelog.

A high Transparency Score is not a certification. It is not a regulatory approval, an audit opinion, or a warranty of safe behavior. It measures what an operator has disclosed and attested to, verified against the artifacts they attached. A published incident report is not a legal finding. It is a corroborated, sourced account made available to the public for scrutiny.

Absence of a report is not evidence of absence — only that no verified report has been received. Readers should combine the registry with their own procurement, audit, and governance processes.