ISO/IEC 27001:2022 provider failover and degradation architecture for enterprise LLM — Reference Architecture

Reference architecture · published 2026-10-16 · regulated enterprise · ISO/IEC 27001:2022 · layout diagram_first · implementation executed in CI before publish

The control that resolves this is a Labelled Degradation Ladder, evaluated before the request reaches a model provider rather than audited afterwards. In regulated enterprise the binding constraint is ISO/IEC 27001:2022, and the question it actually asks is is this risk on the register with a named owner and a treatment. The shape below holds p95 ≤ 600ms over REST over HTTPS, and ships with a standard-library implementation that runs as written.

The request path

Labelled Degradation Ladder — the request path, with the decision points that must exist before the first model call:

flowchart LR
    REQ[Request + min_quality] --> T1{Tier 1 breaker closed?}
    T1 -->|open| T2{Tier 2}
    T1 -->|closed| CALL1[Call tier 1]
    CALL1 -->|fails| BRK[Record failure<br/>maybe open breaker] --> T2
    T2 -->|below min_quality| REFUSE[Refuse: no acceptable tier]
    T2 -->|ok| CALL2[Call tier 2] --> LAB[Label response<br/>degraded=true]

The parts you have to build

1. Quality-ranked tiers — Explicit integer quality per provider, not an implicit order.
2. Circuit breaker — A failing tier leaves rotation instead of being retried into the ground.
3. Half-open probe — One probe after cooldown; recovery is tested, not assumed.
4. Tier label on response — Every answer names its tier and whether it was degraded.
5. Quality floor per caller — A caller declares a minimum; below it is a refusal.
6. Degradation events — Each failover recorded — the signal the incident needs.

Deployment parameters

Every value below is a parameter you can hold a vendor or an internal team to. Adjectives are not acceptance criteria.

The architecture friction point

When the primary provider degrades, traffic fails over to a weaker model and the answers keep flowing. Nobody downstream is told the quality changed, so the incident is discovered in complaints.

What ISO/IEC 27001:2022 specifically requires

ISO 27001 is a management-system standard: the auditor checks that risks are identified, owned, treated and reviewed on a cycle. A perfect technical control that is absent from the risk register is a nonconformity.

The artifact that satisfies it is risk register entries with owners, treatments and review dates. The way teams most often fail it: a technically sound control that no Statement of Applicability references, so it cannot be evidenced.

Working implementation — Degradation ladder with circuit breaker and quality floor

Standard-library Python, no external dependencies, runs as written. Executed by the publish gate: if it raises, this page does not ship. Wire it behind REST over HTTPS; the failure mode it guards against is a retried POST creating a duplicate record. The output you hand the assessor is the Statement of Applicability extract.

"""Degradation ladder with a circuit breaker and labelled quality tiers.

The failure: a provider degrades, traffic silently fails over to a weaker
model, and nobody downstream knows the answer came from a lower tier. The
control is that degradation is explicit and labelled on the response, so a
consumer can refuse a tier it cannot accept, and that a failing provider is
taken out of rotation rather than retried into the ground.
"""
import time


class ProviderDown(Exception):
    pass


class CircuitBreaker:
    def __init__(self, threshold: int = 3, cooldown_s: float = 30.0):
        self.threshold, self.cooldown_s = threshold, cooldown_s
        self.failures, self.opened_at = 0, None

    def record_failure(self, now: float) -> None:
        self.failures += 1
        if self.failures >= self.threshold and self.opened_at is None:
            self.opened_at = now

    def record_success(self) -> None:
        self.failures, self.opened_at = 0, None

    def closed(self, now: float) -> bool:
        if self.opened_at is None:
            return True
        if now - self.opened_at >= self.cooldown_s:
            self.opened_at, self.failures = None, 0   # half-open: allow a probe
            return True
        return False


class Tier:
    def __init__(self, name: str, quality: int, call):
        self.name, self.quality, self.call = name, quality, call
        self.breaker = CircuitBreaker()


class DegradationLadder:
    """Tiers in descending quality. The response always names the tier that
    produced it; a caller declaring min_quality gets a refusal rather than a
    quietly worse answer."""

    def __init__(self, tiers: list):
        self.tiers = sorted(tiers, key=lambda t: -t.quality)
        self.events: list[dict] = []

    def complete(self, prompt: str, min_quality: int = 0, now=None) -> dict:
        now = now if now is not None else time.monotonic()
        attempted = []
        for tier in self.tiers:
            if tier.quality < min_quality:
                continue
            if not tier.breaker.closed(now):
                attempted.append({"tier": tier.name, "skipped": "circuit open"})
                continue
            try:
                text = tier.call(prompt)
            except ProviderDown:
                tier.breaker.record_failure(now)
                attempted.append({"tier": tier.name, "failed": True})
                continue
            tier.breaker.record_success()
            degraded = tier is not self.tiers[0]
            self.events.append({"tier": tier.name, "degraded": degraded})
            return {"text": text, "tier": tier.name, "quality": tier.quality,
                    "degraded": degraded, "attempts": attempted}
        raise ProviderDown(
            f"no tier at or above quality {min_quality} available: {attempted}")


if __name__ == "__main__":
    def primary(p): raise ProviderDown("502 from primary")
    def secondary(p): return f"secondary:{p}"
    def local(p): return f"local-small:{p}"

    ladder = DegradationLadder([
        Tier("frontier", 100, primary),
        Tier("secondary", 70, secondary),
        Tier("local-small", 30, local)])

    r = ladder.complete("summarise", now=0.0)
    assert r["tier"] == "secondary" and r["degraded"] is True, r
    assert r["attempts"][0]["failed"], r

    # A caller that cannot accept degradation gets an error, not a worse answer.
    strict = DegradationLadder([Tier("frontier", 100, primary),
                                Tier("local-small", 30, local)])
    try:
        strict.complete("x", min_quality=90, now=0.0)
        raise AssertionError("served below the declared quality floor")
    except ProviderDown:
        pass

    # The breaker opens after repeated failures and stops the retry storm.
    t = Tier("flaky", 50, primary)
    for i in range(3):
        try:
            t.call("x")
        except ProviderDown:
            t.breaker.record_failure(now=float(i))
    assert not t.breaker.closed(now=3.0), "breaker should be open"
    assert t.breaker.closed(now=40.0), "breaker should half-open after cooldown"

    print("Degradation ladder + circuit breaker: OK")
    print(f"  served tier={r['tier']} degraded={r['degraded']}, "
          f"events {len(ladder.events)}")

The ISO/IEC 27001:2022 control, as code

Statement of Applicability with operating evidence. This is the artifact an assessor asks for under ISO/IEC 27001:2022 specifically — it is not the same control as the one above, and a system that implements one does not thereby implement the other. It executes under the same publish gate.

"""Annex A control to evidence mapping behind a Statement of Applicability.

ISO/IEC 27001:2022 turns on the SoA: every Annex A control is either applicable
with a justification and evidence, or excluded with a justification. The audit
failure is an applicable control whose evidence is a policy document rather
than a record that the control operated.
"""

ANNEX_A_2022 = {
    "A.5.23": "Information security for use of cloud services",
    "A.8.2": "Privileged access rights",
    "A.8.10": "Information deletion",
    "A.8.12": "Data leakage prevention",
    "A.8.15": "Logging",
    "A.8.16": "Monitoring activities",
    "A.8.24": "Use of cryptography",
    "A.8.28": "Secure coding",
}

EVIDENCE_KINDS = {"record", "configuration", "test_result"}
NON_EVIDENCE = {"policy", "intent", "roadmap"}


class StatementOfApplicability:
    def __init__(self, scope: str):
        self.scope = scope
        self.entries: dict[str, dict] = {}

    def apply(self, control: str, justification: str,
              evidence_ref: str, evidence_kind: str) -> None:
        if control not in ANNEX_A_2022:
            raise KeyError(f"{control} is not an Annex A:2022 control")
        if evidence_kind in NON_EVIDENCE:
            raise ValueError(
                f"{control}: {evidence_kind!r} describes intent, not operation")
        if evidence_kind not in EVIDENCE_KINDS:
            raise ValueError(f"{control}: unknown evidence kind {evidence_kind!r}")
        self.entries[control] = {"applicable": True, "justification": justification,
                                 "evidence_ref": evidence_ref,
                                 "evidence_kind": evidence_kind}

    def exclude(self, control: str, justification: str) -> None:
        if control not in ANNEX_A_2022:
            raise KeyError(f"{control} is not an Annex A:2022 control")
        if len(justification.split()) < 5:
            raise ValueError(f"{control}: exclusion needs a real justification")
        self.entries[control] = {"applicable": False, "justification": justification}

    def unaddressed(self) -> list[str]:
        return sorted(set(ANNEX_A_2022) - set(self.entries))

    def certifiable(self) -> dict:
        gaps = self.unaddressed()
        applied = [c for c, e in self.entries.items() if e["applicable"]]
        return {"scope": self.scope, "applied": sorted(applied),
                "excluded": sorted(c for c, e in self.entries.items()
                                   if not e["applicable"]),
                "unaddressed": gaps, "certifiable": not gaps}


if __name__ == "__main__":
    soa = StatementOfApplicability("LLM platform, production tenant")
    soa.apply("A.5.23", "Model inference is a cloud service",
              "cfg://gateway/allowed-providers.yaml", "configuration")
    soa.apply("A.8.2", "Operators hold privileged access to prompts",
              "rec://iam/quarterly-access-review-2026Q3", "record")
    soa.apply("A.8.10", "Retention is enforced on prompt logs",
              "test://retention/expiry-suite-2026-10", "test_result")
    soa.apply("A.8.12", "Egress to model providers can leak data",
              "cfg://dlp/egress-classifier.yaml", "configuration")
    soa.apply("A.8.15", "All inference events are logged",
              "rec://audit/chain-head-2026-10-16", "record")
    soa.apply("A.8.16", "Anomalous usage is monitored",
              "rec://siem/llm-usage-alerts", "record")
    soa.apply("A.8.24", "TLS and at-rest encryption",
              "test://crypto/tls-config-scan", "test_result")
    soa.exclude("A.8.28", "No application code is developed within this scope "
                          "boundary; the platform is configured, not coded")

    report = soa.certifiable()
    assert report["certifiable"], report
    assert len(report["applied"]) == 7 and len(report["excluded"]) == 1, report

    try:
        soa.apply("A.8.15", "we have a logging policy",
                  "doc://policies/logging.pdf", "policy")
        raise AssertionError("accepted a policy document as operating evidence")
    except ValueError:
        pass

    partial = StatementOfApplicability("partial")
    partial.apply("A.8.15", "logging", "rec://x", "record")
    assert not partial.certifiable()["certifiable"]
    assert len(partial.certifiable()["unaddressed"]) == 7

    print("ISO/IEC 27001:2022 SoA evidence mapping: OK")
    print(f"  applied {len(report['applied'])}, excluded {len(report['excluded'])}, "
          f"unaddressed {len(report['unaddressed'])}")

Risk register with acceptance criteria

Each row carries an acceptance parameter rather than an adjective, because "low latency" cannot be signed off. Retention here is bound by per the data class retention schedule, enforced automatically.

Questions to answer before approval

• Where is this control in the risk register and the Statement of Applicability, who owns it, and when was it last reviewed?
  Register entries are generated from the deployed control set, so every control has an SoA reference, a named owner and a review date that cannot silently lapse.
• When the primary provider fails, how does a downstream consumer learn the answer came from a weaker model?
  The response names the tier that produced it and carries a degraded flag. A consumer reading the label can treat a degraded answer differently; silent failover is what this design exists to prevent.
• What stops a failing provider from being retried thousands of times a minute?
  The circuit breaker. After the failure threshold it opens and the tier is skipped entirely until the cooldown elapses, so a failing provider leaves rotation rather than absorbing retries.
• Can a caller that requires top-tier quality be served a degraded answer?
  No. The caller declares a minimum quality and tiers below it are never attempted — the ladder raises instead of serving, which is the correct outcome when no acceptable tier exists.
• How do you know a recovered provider is actually healthy before restoring full traffic?
  The breaker goes half-open after cooldown and admits a single probe. Full rotation is restored only after that probe succeeds, so recovery is tested rather than assumed.
• What prevents failover from moving data outside the contracted region?
  Each tier carries a residency attribute and policy excludes out-of-region tiers from the ladder, so a failover cannot quietly relocate processing.

See also

• Part of enterprise ai compliance architecture.
• HIPAA provider failover and degradation architecture for enterprise LLM applies the same mechanism under a different obligation.
• For the related constraint, see SOC 2 Type II provider failover and degradation architecture.
• This pattern also governs EU AI Act provider failover and degradation architecture for enterprise.
• PCI DSS 4.0 provider failover and degradation architecture for enterprise covers the adjacent case.
• If this is the blocker in front of you: enterprise AI infrastructure review.