1. The question

Search engines and AI systems do not see your brand the way a human does. They see bytes, redirects, HTML, scripts, structured data, and whatever survives policy and rendering. INGEST measures that substrate.

The product question is: can a machine attribute important claims cleanly?

2. Measurement layers

  • Fetch & access — default request plus public crawler identities; status, redirects, body bytes, and challenge signals per agent.
  • Robots policy — matched group/rule and allow/disallow stay separate from HTTP usability. Unreadable robots.txt remains UNKNOWN.
  • Raw HTML vs rendered DOM — title, canonical, H1, links, JSON-LD, and visible text compared; JS-added content measured as dependency.
  • Structured data — JSON-LD parse validity is never treated as schema truth.
  • Atomic facts — propositions preserve subject, predicate, object, source text, assertion mode, polarity, and modality where detected. Interrogatives are not asserted facts.
  • Parity & entities — MATCHED, HUMAN_ONLY, MACHINE_ONLY, CONTRADICTORY, STALE, or AMBIGUOUS. Qualifier/scope differences do not auto-escalate to contradiction.

3. Hard separations

  • Robots policy ≠ HTTP delivery ≠ content usability.
  • JSON parse OK ≠ schema correct.
  • Question ≠ claim.
  • Hash-only noise ≠ differential delivery.
  • UNKNOWN ≠ zero. Unmeasured components stay null.

4. Fail-closed extraction

When extraction is uncertain, INGEST prefers NO_FACT, UNKNOWN, AMBIGUOUS, or INFERRED over an explicit fact that required silent reconstruction.

  1. Correct extraction
  2. Honest under-extraction
  3. Unsupported inference (rejected)

5. Coverage vs score

Measurement coverage reports how many leaf components were actually measured. A composite machine-ingestion score may exist alongside incomplete coverage. A high score with unmeasured freshness or search interpretation does not imply those were observed.

6. Evidence contract

Findings and atomic facts carry inspectable provenance where available: source URL, representation type, selector or location, raw source text, extraction method and assertion mode, and confidence tier. If a claim cannot point back to evidence, it is not treated as trustworthy.

7. Validity & regressions

Permanent fixtures and unit/integration markers lock behavior (for example BCP-47 locale handling, robots group selection, 404 usability, and interrogative-not-fact extraction). Local and CI run npm run fixtures:validity in the INGEST lab. No measurement bug is fixed without a deterministic test.

8. What INGEST does not claim

  • It does not reproduce private crawler internals.
  • It does not decide which conflicting assertion is “true.”
  • It does not treat cloaking labels as legal conclusions.
  • It does not turn marketing language into unsupported service facts.
  • It does not replace Search Console, SERP, or live AI-answer probing (deferred).