HIPAA tracing and token accounting architecture for enterprise LLM deployment — Reference Architecture

Reference architecture · published 2026-10-16 · regulated enterprise · HIPAA · layout diagram_first · implementation executed in CI before publish

The control that resolves this is a Complete-Trace Reconciliation Ledger, evaluated before the request reaches a model provider rather than audited afterwards. In regulated enterprise the binding constraint is HIPAA, and the question it actually asks is can PHI reach a place the BAA does not cover. The shape below holds p95 ≤ 600ms over REST over HTTPS, and ships with a standard-library implementation that runs as written.

Reference architecture

Complete-Trace Reconciliation Ledger — the request path, with the decision points that must exist before the first model call:

flowchart LR
    REQ[Request] --> OPEN[Open trace<br/>model version pinned]
    OPEN --> CTX[Bind prompt digest + sources]
    CTX --> CALL[Model call] --> OUT[Bind output digest + tokens]
    OUT --> LED[(Ledger)]
    BILL[Provider invoice] --> REC{Reconcile}
    LED --> REC
    REC -->|gap| ALERT[Untraced traffic — find the path]
    REC -->|matches| OK[Completeness holds]

Modular implementation framework

1. Per-request trace — Model version, prompt digest, output digest, sources, tokens.
2. Completeness check — A trace missing any required field is reported, not accepted.
3. Source citation list — Which retrieved documents were in context for this answer.
4. Token accounting — Input and output tokens recorded at the call site.
5. Spend reconciliation — Traced tokens against billed tokens; a gap means untraced traffic.
6. Reconstruction API — Given a request id, return everything needed to re-derive the answer.

Technical parameters

Every value below is a parameter you can hold a vendor or an internal team to. Adjectives are not acceptance criteria.

Where the rollout actually stalls

An answer is disputed weeks after it was given and nobody can say which model version, which documents and which prompt produced it. Traces exist, but they are sampled, so the disputed request is the one that was not kept.

What HIPAA specifically requires

HIPAA holds the covered entity responsible for PHI wherever it flows. A model provider without a signed BAA is not a permissible destination, and 'the prompt only contained a little PHI' is not a defence — the minimum necessary standard is itself a control.

The artifact that satisfies it is a de-identification record per request plus the BAA chain. The way teams most often fail it: PHI reaching a subprocessor with no BAA, via a logging sidecar nobody mapped.

The implementation — Trace completeness with spend reconciliation

Standard-library Python, no external dependencies, runs as written. Executed by the publish gate: if it raises, this page does not ship. Wire it behind REST over HTTPS; the failure mode it guards against is a retried POST creating a duplicate record. The output you hand the assessor is the minimum-necessary de-identification log.

"""Trace completeness: every response tied to the inputs that produced it.

The failure: an answer is disputed weeks later and nobody can reconstruct which
model version, which retrieved documents and which prompt produced it. Sampling
traces at 1% guarantees the disputed one is missing, so completeness is the
property to enforce — and untraced spend is the signal that it has broken.
"""
import hashlib


class Trace:
    def __init__(self, request_id: str, model_version: str):
        self.request_id = request_id
        self.model_version = model_version
        self.sources: list[str] = []
        self.prompt_digest = None
        self.output_digest = None
        self.tokens = {"in": 0, "out": 0}

    def bind_prompt(self, prompt: str) -> None:
        self.prompt_digest = hashlib.sha256(prompt.encode()).hexdigest()[:16]

    def bind_output(self, output: str) -> None:
        self.output_digest = hashlib.sha256(output.encode()).hexdigest()[:16]

    def cite(self, source_id: str) -> None:
        self.sources.append(source_id)

    def complete(self) -> bool:
        return all([self.prompt_digest, self.output_digest,
                    self.model_version, self.tokens["in"] > 0])

    def missing(self) -> list[str]:
        gaps = []
        if not self.prompt_digest: gaps.append("prompt_digest")
        if not self.output_digest: gaps.append("output_digest")
        if not self.model_version: gaps.append("model_version")
        if self.tokens["in"] <= 0: gaps.append("token_accounting")
        return gaps


class Ledger:
    """Reconciles traced spend against billed spend. A gap means requests are
    escaping the traced path, which is the thing to find — not a rounding
    error to tolerate."""

    def __init__(self, tolerance: float = 0.01):
        self.traces: dict[str, Trace] = {}
        self.tolerance = tolerance

    def open(self, request_id: str, model_version: str) -> Trace:
        t = Trace(request_id, model_version)
        self.traces[request_id] = t
        return t

    def reconstruct(self, request_id: str) -> dict | None:
        t = self.traces.get(request_id)
        if not t:
            return None
        return {"request_id": t.request_id, "model_version": t.model_version,
                "prompt_digest": t.prompt_digest, "output_digest": t.output_digest,
                "sources": list(t.sources), "tokens": dict(t.tokens),
                "complete": t.complete(), "missing": t.missing()}

    def traced_tokens(self) -> int:
        return sum(t.tokens["in"] + t.tokens["out"] for t in self.traces.values())

    def reconcile(self, billed_tokens: int) -> dict:
        traced = self.traced_tokens()
        gap = billed_tokens - traced
        ratio = gap / billed_tokens if billed_tokens else 0.0
        return {"traced": traced, "billed": billed_tokens, "gap": gap,
                "untraced_ratio": round(ratio, 4),
                "within_tolerance": abs(ratio) <= self.tolerance,
                "incomplete_traces": [r for r, t in self.traces.items()
                                      if not t.complete()]}


if __name__ == "__main__":
    led = Ledger()
    t = led.open("req-1", "gpt-x@2026-09")
    t.bind_prompt("What is our refund policy?")
    t.cite("doc://policies/refunds#v7")
    t.tokens = {"in": 820, "out": 240}
    t.bind_output("Refunds are processed within 14 days.")

    rec = led.reconstruct("req-1")
    assert rec["complete"], rec
    assert rec["sources"] == ["doc://policies/refunds#v7"], rec
    assert rec["model_version"] == "gpt-x@2026-09"

    # An incomplete trace must be visible, not silently acceptable.
    t2 = led.open("req-2", "gpt-x@2026-09")
    t2.tokens = {"in": 100, "out": 50}
    assert not t2.complete() and "prompt_digest" in t2.missing(), t2.missing()

    clean = led.reconcile(billed_tokens=1210)
    assert clean["within_tolerance"], clean
    assert clean["incomplete_traces"] == ["req-2"], clean

    # Requests escaping the traced path show up as an untraced gap.
    leaky = led.reconcile(billed_tokens=4000)
    assert not leaky["within_tolerance"], leaky
    assert leaky["untraced_ratio"] > 0.6, leaky

    assert led.reconstruct("req-missing") is None

    print("Trace completeness ledger: OK")
    print(f"  reconstructed req-1 complete={rec['complete']}, "
          f"untraced ratio clean={clean['untraced_ratio']} "
          f"leaky={leaky['untraced_ratio']}")

The HIPAA control, as code

Minimum-necessary de-identification with a BAA-pinned egress gate. This is the artifact an assessor asks for under HIPAA specifically — it is not the same control as the one above, and a system that implements one does not thereby implement the other. It executes under the same publish gate.

"""Minimum-necessary de-identification with a BAA-pinned destination gate.

HIPAA holds the covered entity responsible for PHI wherever it flows, so the
control is not "we told people not to paste PHI". Two things must be
structural: PHI is reduced to the minimum necessary BEFORE egress, and the
destination set is pinned to those under a signed BAA, failing closed on
anything else.

Safe Harbor (45 CFR 164.514(b)(2)) lists 18 identifier classes. This implements
the mechanical subset and records a per-request de-identification receipt,
because the auditor asks for proof per request, not a policy document.
"""
import re

SAFE_HARBOR_PATTERNS = {
    "name_mrn": re.compile(r"\bMRN[:\s#]*([A-Z0-9-]{4,})", re.I),
    "ssn": re.compile(r"\b\d{3}-\d{2}-\d{4}\b"),
    "phone": re.compile(r"\b(?:\+1[-.\s]?)?\(?\d{3}\)?[-.\s]?\d{3}[-.\s]?\d{4}\b"),
    "email": re.compile(r"\b[\w.+-]+@[\w-]+\.[\w.]+\b"),
    "date_full": re.compile(r"\b\d{1,2}/\d{1,2}/(19|20)\d{2}\b"),
    "member_id": re.compile(r"\b(?:member|policy)\s*(?:id|#)[:\s]*([A-Z0-9-]{5,})", re.I),
}


class BAARegistry:
    """Destinations are permissible only as attested data, never by convention."""

    def __init__(self):
        self._signed: dict[str, str] = {}

    def attest(self, destination: str, agreement_ref: str) -> None:
        self._signed[destination] = agreement_ref

    def covered(self, destination: str) -> bool:
        return destination in self._signed

    def reference(self, destination: str) -> str | None:
        return self._signed.get(destination)


class PHIEgressGate:
    def __init__(self, registry: BAARegistry):
        self.registry = registry
        self.receipts: list[dict] = []

    def deidentify(self, text: str) -> tuple[str, dict[str, int]]:
        counts: dict[str, int] = {}
        out = text
        for label, pattern in SAFE_HARBOR_PATTERNS.items():
            out, n = pattern.subn(f"[REDACTED:{label}]", out)
            if n:
                counts[label] = n
        return out, counts

    def send(self, text: str, destination: str) -> dict:
        """Refuses rather than redacting-and-hoping when the BAA is absent."""
        redacted, counts = self.deidentify(text)
        phi_present = bool(counts)
        if phi_present and not self.registry.covered(destination):
            receipt = {"destination": destination, "admitted": False,
                       "reason": "no BAA on file for a PHI-classified payload",
                       "identifier_classes": sorted(counts)}
            self.receipts.append(receipt)
            raise PermissionError(receipt["reason"])
        receipt = {
            "destination": destination,
            "admitted": True,
            "baa_ref": self.registry.reference(destination),
            "identifier_classes": sorted(counts),
            "redactions": sum(counts.values()),
            "payload": redacted,
        }
        self.receipts.append(receipt)
        return receipt


if __name__ == "__main__":
    reg = BAARegistry()
    reg.attest("azure-openai-us-east", "BAA-2026-0041")
    gate = PHIEgressGate(reg)

    note = ("Patient MRN: 88213A, contact 555-867-5309 or a.patient@example.com, "
            "DOB 04/11/1974, follow-up scheduled.")
    ok = gate.send(note, "azure-openai-us-east")
    assert ok["admitted"] and ok["redactions"] >= 4, ok
    assert "88213A" not in ok["payload"] and "555-867-5309" not in ok["payload"], ok
    assert "a.patient@example.com" not in ok["payload"], ok

    try:
        gate.send(note, "some-vendor-eval-endpoint")
        raise AssertionError("PHI reached a destination with no BAA")
    except PermissionError:
        pass

    # No PHI means no BAA requirement — the gate must not over-block.
    benign = gate.send("Summarise ward occupancy trends.", "some-vendor-eval-endpoint")
    assert benign["admitted"] and benign["redactions"] == 0, benign

    print("HIPAA minimum-necessary egress gate: OK")
    print(f"  receipts {len(gate.receipts)}, "
          f"refusals {sum(1 for r in gate.receipts if not r['admitted'])}")

Risk register with acceptance criteria

Each row carries an acceptance parameter rather than an adjective, because "low latency" cannot be signed off. Retention here is bound by per the data class retention schedule, enforced automatically.

Questions to answer before approval

• Which destinations hold a BAA, and what structurally prevents PHI from reaching one that does not?
  Destinations carry a BAA flag as attested data, and the routing gate refuses a PHI-classified payload to any destination without it. The refusal is logged with its reason.
• A customer disputes an answer from six weeks ago. What can you reconstruct?
  The model version, prompt digest, output digest, the retrieved sources in context, and token counts — enough to re-derive the answer rather than argue about it.
• How would you know if some requests never reach the traced path?
  Reconciliation. Traced token spend is compared against the provider invoice; a gap beyond tolerance means traffic is escaping the traced path, and that gap is the thing to chase.
• Which model version produced a specific output, and how is that known rather than inferred?
  It is pinned on the trace when the trace opens, before the call. Inferring it afterwards from deployment history is exactly how the wrong version gets attributed.
• Does the trace store the raw prompt, and what does that mean for personal data?
  Digests by default, with raw retained only where there is a lawful basis. The digest is enough to prove which prompt produced which output without storing the text.
• What does tracing add to the latency of the request it traces?
  Nothing on the critical path: trace writes are buffered and asynchronous, and the added p95 is measured against the agreed budget rather than assumed to be negligible.

Adjacent problems

• Part of enterprise ai compliance architecture.
• This pattern also governs HIPAA cost control architecture for enterprise LLM deployment.
• HIPAA provider failover and degradation architecture for enterprise LLM covers the adjacent case.
• HIPAA prompt injection defence architecture for enterprise LLM deployment applies the same mechanism under a different obligation.
• Where this interacts with another regime, see HIPAA legacy integration patterns for enterprise AI agents.
• If this is the blocker in front of you: enterprise AI infrastructure review.