Policy Shadow Mode

Policy Shadow Mode

Test new policies against live production traffic without affecting any decisions. Shadow policies are evaluated on every incoming action but their verdicts are logged separately, the actual authorisation decision is never changed.

Info

Use case: Before deploying a policy that blocks all trades over $100K, run it in shadow mode for 24 hours. The Policy Simulation dashboard shows exactly which actions would have been blocked, which agents would be affected, and whether any outcomes would change, all with zero production risk.

How It Works

Step	What Happens
1. Create	Create a policy with "mode": "shadow". It's active but observe-only.
2. Evaluate	Every /enforce/intercept call evaluates shadow policies in a separate pass after the real decision.
3. Record	Shadow results are stored in shadow_results on the decision record: what the shadow policy would have decided, which policies triggered, whether the outcome would differ.
4. Report	The /enforce/shadow/report endpoint and the Policy Simulation dashboard aggregate the impact.
5. Promote	When satisfied, promote to live with a single API call or one click in the dashboard.

Create a Shadow Policy

resp = requests.post(f"{BASE}/enforce/policies", headers=HEADERS, json={
    "name": "Block Trades Over 500K",
    "policy_type": "threshold",
    "action_types": ["execute_trade"],
    "decision": "block",
    "priority": 200,
    "trust_threshold": 95,
    "mode": "shadow"        # ← observe-only
})
# resp.json()["policy"]["mode"] → "shadow"

Shadow Results on Decisions

After sending traffic, each decision record includes a shadow_results field:

{
  "decision": "allow",
  "shadow_results": {
    "shadow_decision": "block",
    "would_have_changed": true,
    "live_decision": "allow",
    "triggered": [
      {
        "policy_id": "abc123",
        "name": "Block Trades Over 500K",
        "decision": "block",
        "reason": "Agent trust 50.2 < threshold 95"
      }
    ],
    "reasoning": "Agent trust 50.2 < threshold 95"
  }
}

GET /v1/enforce/shadow/report

Aggregate impact report for all active shadow policies over a time window.

curl -H "X-API-Key: xb_..." \
  "https://xybern.com/api/v1/enforce/shadow/report?hours=24"

{
  "shadow_policies": 2,
  "hours": 24,
  "report": {
    "total_evaluated": 847,
    "would_block": 23,
    "would_escalate": 8,
    "would_change_outcome": 15,
    "affected_agents": ["agent_abc", "agent_def"],
    "per_policy": [
      {
        "policy_id": "abc123",
        "name": "Block Trades Over 500K",
        "would_block": 23,
        "would_escalate": 0,
        "total_triggered": 23
      }
    ]
  }
}

POST /v1/enforce/shadow/{policy_id}/promote

Promote a shadow policy to live mode, it will start enforcing immediately.

resp = requests.post(
    f"{BASE}/enforce/shadow/{policy_id}/promote",
    headers=HEADERS
)
# resp.json()["policy"]["mode"] → "live"

Toggle Mode via Update

You can also switch any policy between live and shadow using the standard update endpoint:

requests.put(f"{BASE}/enforce/policies/{policy_id}",
    headers=HEADERS,
    json={"mode": "shadow"}  # or "live"
)

Dashboard

The Authorisation Layer dashboard includes:

• Policies tab, every policy shows a LIVE or SHADOW badge with a one-click mode toggle.
• Policy Simulation tab, impact stats (would-block, would-escalate, outcome changes), per-policy breakdown, live vs shadow comparison table, and a "Promote to Live" button.