SOC 2 Type II tracing and token accounting architecture for enterprise LLM deployment — Reference Architecture

Reference architecture · published 2026-10-16 · regulated enterprise · SOC 2 Type II · layout diagram_first · implementation executed in CI before publish

The control that resolves this is a Complete-Trace Reconciliation Ledger, evaluated before the request reaches a model provider rather than audited afterwards. In regulated enterprise the binding constraint is SOC 2 Type II, and the question it actually asks is did this control operate effectively for the whole observation window. The shape below holds p95 ≤ 600ms over REST over HTTPS, and ships with a standard-library implementation that runs as written.

System architecture

Complete-Trace Reconciliation Ledger — the request path, with the decision points that must exist before the first model call:

flowchart LR
    REQ[Request] --> OPEN[Open trace<br/>model version pinned]
    OPEN --> CTX[Bind prompt digest + sources]
    CTX --> CALL[Model call] --> OUT[Bind output digest + tokens]
    OUT --> LED[(Ledger)]
    BILL[Provider invoice] --> REC{Reconcile}
    LED --> REC
    REC -->|gap| ALERT[Untraced traffic — find the path]
    REC -->|matches| OK[Completeness holds]

Modular implementation framework

1. Per-request trace — Model version, prompt digest, output digest, sources, tokens.
2. Completeness check — A trace missing any required field is reported, not accepted.
3. Source citation list — Which retrieved documents were in context for this answer.
4. Token accounting — Input and output tokens recorded at the call site.
5. Spend reconciliation — Traced tokens against billed tokens; a gap means untraced traffic.
6. Reconstruction API — Given a request id, return everything needed to re-derive the answer.

Technical parameters

Every value below is a parameter you can hold a vendor or an internal team to. Adjectives are not acceptance criteria.

Why this blocks production

An answer is disputed weeks after it was given and nobody can say which model version, which documents and which prompt produced it. Traces exist, but they are sampled, so the disputed request is the one that was not kept.

What SOC 2 Type II specifically requires

SOC 2 Type II is not a point-in-time assertion. The auditor samples evidence across a 3-12 month observation window and asks whether the control operated on every day of it, not whether it exists today.

The artifact that satisfies it is a sampled, time-ordered population the auditor can draw from. The way teams most often fail it: the control was added mid-window, so the population has a hole and the observation period restarts.

Reference implementation — Trace completeness with spend reconciliation

Standard-library Python, no external dependencies, runs as written. Executed by the publish gate: if it raises, this page does not ship. Wire it behind REST over HTTPS; the failure mode it guards against is a retried POST creating a duplicate record. The output you hand the assessor is the observation-window evidence export.

"""Trace completeness: every response tied to the inputs that produced it.

The failure: an answer is disputed weeks later and nobody can reconstruct which
model version, which retrieved documents and which prompt produced it. Sampling
traces at 1% guarantees the disputed one is missing, so completeness is the
property to enforce — and untraced spend is the signal that it has broken.
"""
import hashlib


class Trace:
    def __init__(self, request_id: str, model_version: str):
        self.request_id = request_id
        self.model_version = model_version
        self.sources: list[str] = []
        self.prompt_digest = None
        self.output_digest = None
        self.tokens = {"in": 0, "out": 0}

    def bind_prompt(self, prompt: str) -> None:
        self.prompt_digest = hashlib.sha256(prompt.encode()).hexdigest()[:16]

    def bind_output(self, output: str) -> None:
        self.output_digest = hashlib.sha256(output.encode()).hexdigest()[:16]

    def cite(self, source_id: str) -> None:
        self.sources.append(source_id)

    def complete(self) -> bool:
        return all([self.prompt_digest, self.output_digest,
                    self.model_version, self.tokens["in"] > 0])

    def missing(self) -> list[str]:
        gaps = []
        if not self.prompt_digest: gaps.append("prompt_digest")
        if not self.output_digest: gaps.append("output_digest")
        if not self.model_version: gaps.append("model_version")
        if self.tokens["in"] <= 0: gaps.append("token_accounting")
        return gaps


class Ledger:
    """Reconciles traced spend against billed spend. A gap means requests are
    escaping the traced path, which is the thing to find — not a rounding
    error to tolerate."""

    def __init__(self, tolerance: float = 0.01):
        self.traces: dict[str, Trace] = {}
        self.tolerance = tolerance

    def open(self, request_id: str, model_version: str) -> Trace:
        t = Trace(request_id, model_version)
        self.traces[request_id] = t
        return t

    def reconstruct(self, request_id: str) -> dict | None:
        t = self.traces.get(request_id)
        if not t:
            return None
        return {"request_id": t.request_id, "model_version": t.model_version,
                "prompt_digest": t.prompt_digest, "output_digest": t.output_digest,
                "sources": list(t.sources), "tokens": dict(t.tokens),
                "complete": t.complete(), "missing": t.missing()}

    def traced_tokens(self) -> int:
        return sum(t.tokens["in"] + t.tokens["out"] for t in self.traces.values())

    def reconcile(self, billed_tokens: int) -> dict:
        traced = self.traced_tokens()
        gap = billed_tokens - traced
        ratio = gap / billed_tokens if billed_tokens else 0.0
        return {"traced": traced, "billed": billed_tokens, "gap": gap,
                "untraced_ratio": round(ratio, 4),
                "within_tolerance": abs(ratio) <= self.tolerance,
                "incomplete_traces": [r for r, t in self.traces.items()
                                      if not t.complete()]}


if __name__ == "__main__":
    led = Ledger()
    t = led.open("req-1", "gpt-x@2026-09")
    t.bind_prompt("What is our refund policy?")
    t.cite("doc://policies/refunds#v7")
    t.tokens = {"in": 820, "out": 240}
    t.bind_output("Refunds are processed within 14 days.")

    rec = led.reconstruct("req-1")
    assert rec["complete"], rec
    assert rec["sources"] == ["doc://policies/refunds#v7"], rec
    assert rec["model_version"] == "gpt-x@2026-09"

    # An incomplete trace must be visible, not silently acceptable.
    t2 = led.open("req-2", "gpt-x@2026-09")
    t2.tokens = {"in": 100, "out": 50}
    assert not t2.complete() and "prompt_digest" in t2.missing(), t2.missing()

    clean = led.reconcile(billed_tokens=1210)
    assert clean["within_tolerance"], clean
    assert clean["incomplete_traces"] == ["req-2"], clean

    # Requests escaping the traced path show up as an untraced gap.
    leaky = led.reconcile(billed_tokens=4000)
    assert not leaky["within_tolerance"], leaky
    assert leaky["untraced_ratio"] > 0.6, leaky

    assert led.reconstruct("req-missing") is None

    print("Trace completeness ledger: OK")
    print(f"  reconstructed req-1 complete={rec['complete']}, "
          f"untraced ratio clean={clean['untraced_ratio']} "
          f"leaky={leaky['untraced_ratio']}")

The SOC 2 Type II control, as code

Type II evidence sampling over the observation window. This is the artifact an assessor asks for under SOC 2 Type II specifically — it is not the same control as the one above, and a system that implements one does not thereby implement the other. It executes under the same publish gate.

"""SOC 2 Type II evidence sampling over an observation window.

Type II is not "the control exists" — it is "the control operated throughout
the period". An auditor samples days from the window and asks for proof the
control ran on each sampled day. A gap of one day with no evidence is an
exception, so the population must be built from operational records rather
than asserted.
"""
import hashlib
from datetime import date, timedelta


class ObservationWindow:
    """The population an auditor samples from, plus deterministic selection."""

    def __init__(self, start: date, end: date, control_id: str):
        if end < start:
            raise ValueError("window end precedes start")
        self.start, self.end, self.control_id = start, end, control_id
        self._evidence: dict[date, list[str]] = {}

    def record(self, day: date, artifact_ref: str) -> None:
        if not (self.start <= day <= self.end):
            raise ValueError(f"{day} outside observation window")
        self._evidence.setdefault(day, []).append(artifact_ref)

    def population(self) -> list[date]:
        n = (self.end - self.start).days + 1
        return [self.start + timedelta(days=i) for i in range(n)]

    def gaps(self) -> list[date]:
        """Days with no evidence. Each one is a Type II exception."""
        return [d for d in self.population() if not self._evidence.get(d)]

    def sample(self, size: int) -> list[date]:
        """Deterministic, reproducible selection — an auditor must be able to
        re-derive the same sample from the same seed rather than trust ours."""
        pop = self.population()
        size = min(size, len(pop))
        seed = f"{self.control_id}:{self.start}:{self.end}"
        scored = sorted(
            pop, key=lambda d: hashlib.sha256(f"{seed}:{d}".encode()).hexdigest())
        return sorted(scored[:size])

    def evidence_for(self, day: date) -> list[str]:
        return list(self._evidence.get(day, []))

    def attestation(self, size: int = 25) -> dict:
        sampled = self.sample(size)
        missing = [d for d in sampled if not self._evidence.get(d)]
        return {
            "control_id": self.control_id,
            "window": f"{self.start}..{self.end}",
            "population_days": len(self.population()),
            "sampled_days": len(sampled),
            "exceptions": [str(d) for d in missing],
            "opinion": "unqualified" if not missing else "qualified",
        }


if __name__ == "__main__":
    start, end = date(2026, 1, 1), date(2026, 3, 31)
    w = ObservationWindow(start, end, "CC6.1-tenant-isolation")
    for d in w.population():
        w.record(d, f"s3://evidence/{d}/isolation-check.json")
    clean = w.attestation()
    assert clean["opinion"] == "unqualified", clean
    assert clean["population_days"] == 90, clean

    # One missing day is one exception — and it must survive sampling to matter.
    w2 = ObservationWindow(start, end, "CC6.1-tenant-isolation")
    sampled = set(w2.sample(25))
    for d in w2.population():
        if d != sorted(sampled)[0]:
            w2.record(d, "ok")
    assert w2.attestation()["opinion"] == "qualified", w2.attestation()
    assert len(w2.gaps()) == 1, w2.gaps()

    # The sample must be reproducible from the same seed.
    assert w.sample(25) == ObservationWindow(start, end,
                                             "CC6.1-tenant-isolation").sample(25)
    print("SOC 2 Type II evidence sampling: OK")
    print(f"  population {clean['population_days']} days, "
          f"sample {clean['sampled_days']}, opinion {clean['opinion']}")

Risk and control matrix

Each row carries an acceptance parameter rather than an adjective, because "low latency" cannot be signed off. Retention here is bound by per the data class retention schedule, enforced automatically.

Control questions before sign-off

• Show me the control-execution population for the full window, and the days where it is empty.
  Every request emits an execution record, so the population is continuous by construction. A gap is an alertable incident, not something discovered during the audit.
• A customer disputes an answer from six weeks ago. What can you reconstruct?
  The model version, prompt digest, output digest, the retrieved sources in context, and token counts — enough to re-derive the answer rather than argue about it.
• How would you know if some requests never reach the traced path?
  Reconciliation. Traced token spend is compared against the provider invoice; a gap beyond tolerance means traffic is escaping the traced path, and that gap is the thing to chase.
• Which model version produced a specific output, and how is that known rather than inferred?
  It is pinned on the trace when the trace opens, before the call. Inferring it afterwards from deployment history is exactly how the wrong version gets attributed.
• Does the trace store the raw prompt, and what does that mean for personal data?
  Digests by default, with raw retained only where there is a lawful basis. The digest is enough to prove which prompt produced which output without storing the text.
• What does tracing add to the latency of the request it traces?
  Nothing on the critical path: trace writes are buffered and asynchronous, and the added p95 is measured against the agreed budget rather than assumed to be negligible.

Related architectures

• Part of enterprise ai compliance architecture.
• SOC 2 Type II reference architecture for enterprise LLM deployment covers the adjacent case.
• The same control appears in SOC 2 Type II agent permission and delegated access architecture.
• A different failure mode of the same component: SOC 2 Type II audit evidence architecture for enterprise LLM deployment.
• Where this interacts with another regime, see SOC 2 Type II model release gate and golden set evaluation for enterprise.
• If this is the blocker in front of you: enterprise AI infrastructure review.