Agent action gating

Before an agent triggers side effects (refunds, emails, purchases), require calibrated probability + uncertainty to meet your safety bar.

Overview

Problem

Agents often act on overconfident but wrong judgments. A raw score can look “safe” even when the true probability is borderline.

Reality Signal

Convert a score into prob_est + uncertainty and a boolean decision.

Policy

Act only when decision=true and uncertainty is below your guardrail. Otherwise reprompt with constraints, reroute to a stronger model, or escalate to a human approver.

Architecture

Base model produces a score for the decision you care about.
Send the score to /decide.
Use prob_est + uncertainty to route safely.
When the true outcome is known, call /feedback to improve calibration.

Routing actions: automate, reroute, reprompt, or escalate.

1) Decide before acting

Call /decide with your model’s score. Route based on calibrated probability and uncertainty.

Request

bash

curl -X POST https://onprem-api-sowl.jollysand-1b9ed42e.swedencentral.azurecontainerapps.io/decide \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "features": { "score": 0.82 }
  }'

Response

json

{
  "decision_id": 123,
  "prob_est": 0.62,
  "uncertainty": 0.08,
  "decision": false
}

Use prob_est + uncertainty as routing signals.

2) Close the loop with feedback

When ground truth is known, send /feedback with the decision_id so calibration improves over time.

Feedback (cURL)

bash

curl -X POST https://onprem-api-sowl.jollysand-1b9ed42e.swedencentral.azurecontainerapps.io/feedback \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "decision_id": 123,
    "feedback": 1,
    "force_retrain": false
  }'

Reference implementation

Python

python

import requests

API_URL = "https://onprem-api-sowl.jollysand-1b9ed42e.swedencentral.azurecontainerapps.io"
HEADERS = {"x-api-key": "YOUR_API_KEY"}

def rc_decide(score: float):
    payload = {"features": {"score": score}
    r = requests.post(f"{API_URL}/decide", json=payload, headers=HEADERS)
    r.raise_for_status()
    return r.json()

action_score = 0.81
rc = rc_decide(action_score)

if rc["decision"] and rc["uncertainty"] <= 0.15:
    perform_refund()
else:
    reprompt_with_constraints()
    # or: reroute_to_stronger_model()
    # or: request_human_approval()

Tip: for high‑stakes actions, use a stricter uncertainty guardrail (e.g., 0.10) and log all low‑confidence actions for review.