Decision Traces in Generative Search

Definition

What is a decision trace?

A decision trace is the inspectable record connecting an observation to the evidence used, the interpretation applied, the decision reached, the action taken, and the verification that followed.

When the subject is an opaque retrieval or generative system, its private reasoning is unavailable. NRLC therefore treats repeated outcomes under controlled variation as evidence for an inference—not as access to the system’s hidden reasoning. The trace documents NRLC’s reasoning so another person can inspect it, challenge it, and repeat the test.

Reasoning discipline

Observation ≠ interpretation ≠ claim ≠ decision

A diagnostic can observe disagreement. It cannot infer Google’s selected canonical, indexing state, rankings, traffic, or AI citation behavior from that disagreement.

Worked example

Canonical consistency test

This synthetic example demonstrates the procedure used by the public Search System Diagnostic. It is not a finding from a named site.

1. 01

   Observation

   Three public representations disagree on canonical identity.

   This statement records the directly comparable condition. It does not yet explain the cause or downstream effect.

2. 02

   Evidence

   Freeze the values exactly as observed.

   HTML canonical

   https://example.com/product/x

   JSON-LD URL

   https://example.com/products/x

   Sitemap URL

   https://example.com/product/x/

   Preserve the response, extraction timestamp, requested URL, final URL, and normalization rule. A screenshot alone is insufficient evidence for a machine-readable value.

3. 03

   Interpretation

   Multiple public representations identify different URLs as the resource.

   The interpretation follows only after resolving harmless syntax differences. A trailing slash may normalize to the same resource; singular and plural paths do not normalize away.

4. 04

   Decision

   Canonical identity is not sufficiently consistent to PASS.

   The predeclared rule is: explicit, material disagreement between representations is FAIL; unavailable or ambiguous evidence is REVIEW; agreement after documented normalization is PASS.

5. 05

   Action

   Align representations from one approved canonical source.

   Choose the intended resource identity, then align HTML canonical metadata, page JSON-LD, sitemap membership, and internal references. If a registry owns these values, correct the registry projection rather than patching each output independently.

6. 06

   Verification

   Re-run the same frozen test against the changed resource.

   PASS only when the same extraction and normalization procedure returns one canonical identity across the inspected representations. Record unresolved differences as REVIEW or FAIL; do not substitute a new test because it produces a cleaner result.

Classification boundary

When a binary decision is justified

A specific observed output can often be classified as present or absent: a segment was included in a captured answer, a canonical value matched, or an expected resource was omitted. That binary observation is not a universal visibility score and does not reveal the system’s confidence or private reasoning.

Across repeated, controlled observations, recurrence may support an inference about the conditions associated with inclusion or exclusion. The inference must remain separate from each observed outcome and must be revised when the pattern does not recur.

Reproducible procedure

How to create a decision trace

1. Define the question. Name one condition that can be observed and one decision rule that can classify it.
2. Freeze the procedure. Specify inputs, surfaces, extraction method, normalization, and PASS / REVIEW / FAIL criteria before seeing the changed result.
3. Capture observations. Preserve raw values, source locations, timestamps, requested and final URLs, and unavailable evidence. Do not replace missing evidence with zero.
4. Separate interpretation. Write the raw observation first. Then state the rule or model used to explain its significance.
5. Test alternatives. Check redirects, URL normalization, environment, transient fetch failures, and other explanations that could produce the same observation.
6. Make the decision. Apply the frozen rule. REVIEW is the correct result when evidence is insufficient or ambiguous.
7. Change one condition. Record the exact action and avoid bundling unrelated changes when attribution matters.
8. Verify by repetition. Re-run the same procedure, preserve the new evidence, and compare it with the frozen baseline.

Decision contract

Rules for the canonical consistency example

These statuses describe the observed consistency condition. They are not scores and do not predict indexing, ranking, traffic, retrieval, or citation outcomes.

Reusable record

Minimum decision-trace record

question
procedure_version
observed_at
requested_url
final_url
observation
evidence[]
normalization_rule
interpretation
alternative_explanations[]
decision_rule
decision
action
verification_observation
limitations

A complete record lets another reviewer distinguish what was observed from what NRLC inferred and decided.

Limits and falsification

What this method cannot prove

• A public-output trace does not expose a search engine or model’s private reasoning.
• Representation agreement does not prove Google indexation, ranking improvement, retrieval, or citation.
• A single before/after result does not establish causation when other conditions changed.
• Repeated outcomes support an inference only when the variation, environment, and observation window are documented.

The interpretation fails when the same controlled procedure repeatedly produces evidence that contradicts its decision rule, when alternative explanations account for the observation better, or when the predicted outcome does not recur after the proposed condition is changed.