Blog / Topic
Runtime control
Start here
All articles
What controls do AI agents in procurement need?
PO thresholds, no vendor bank-detail changes, three-way match and separation of duties: a control checklist for procurement AI agents.
Should AI agent guardrails be rules or models?
Deterministic rules, small classifiers and LLM judges compared on latency, cost, explainability and false positives, plus a layered guardrail pattern.
What timeouts should you set for AI agent tool calls?
A timeout policy for AI agent tool calls: per-tool-class limits, total task budgets, partial results, retries with backoff and MCP cancellation.
How do you detect an AI agent trying to expand its own permissions?
Signals that an AI agent is escalating its own access: IAM API calls, OAuth scope requests, config edits, new credentials. Detection rules and responses.
How do idempotency keys stop AI agents repeating actions?
Retries at the model, agent and gateway layers can make AI agents create duplicates. How idempotency keys and idempotent tool design prevent it.
How do you detect an AI agent acting as a user without saying so?
Signals that reveal an AI agent using a person's account: token type, client ID, timing, API path. Plus the policy that requires agents to carry their own identity.
How do you set content safety thresholds for customer-facing AI agents?
Set harm thresholds per channel and audience using category and severity scales, test them on real traffic, and review false positives weekly.
When should a customer service AI agent hand off to a human?
A numbered checklist of handoff triggers for customer service AI agents, the context to pass to the human, and how to measure handoff quality.
How do you measure and tune guardrail false positives?
A step-by-step loop for measuring AI guardrail false positives: labelled samples, precision and recall per policy, monitor mode, thresholds and appeals.
What is dry-run mode for AI agents, and when should you use it?
Dry-run mode lets an AI agent plan real actions without executing them. How it differs from shadow mode, how to build it and when to exit it.
What happens when an AI agent invents a tool or its arguments?
Why agents call tools that do not exist or pass wrong arguments, what strict tool schemas guarantee, and the server-side checks and logs you still need.
Should AI agents be restricted to business hours or change windows?
Some agent actions should only run when people can respond. A decision table by action type, timezone pitfalls, an on-call rule and exception handling.
How do you scan AI agent outputs for sensitive data before they leave?
Output-side DLP for AI agents: which detectors to use, where to place the scan, when to redact versus block, and how to handle false positives.
How do you run a new AI agent in shadow mode before it acts?
Shadow mode lets a new AI agent see real inputs while its write actions are stubbed. How to design it, compare it with human outcomes, and set exit criteria.
How do you monitor hand-offs between agents in a multi-agent system?
Monitor multi-agent hand-offs with one trace across agents, a fixed set of fields per hand-off, and detections for loops, lost context and privilege escalation.
How do you control AI agents that can move money?
Controls for AI agents with payment, refund or transfer tools: amount caps, payee allow-lists, velocity limits, dual approval and reconciliation evidence.
Which AI agent events should go to your SIEM?
A prioritised list of AI agent events for your SIEM, with the fields to include, volume guidance and example detections your SOC can use.
How do you apply guardrails to streaming LLM responses?
Streaming lets content reach users before a guardrail check ends. Buffered vs asynchronous modes, real vendor behaviour, and when to turn streaming off.
Should AI agents follow change freezes?
Prompts, models, tools and policies are all changes to an AI agent. A freeze scope table, plus how to handle provider model retirements and emergencies.
Should AI guardrails fail open or fail closed?
When a guardrail service times out or goes down, should agent traffic pass or stop? A decision table by risk tier and action type, with timeouts and fallbacks.
How do you test a new AI agent policy before enforcing it?
Run new agent policies in monitor mode on real traffic, measure the false-positive rate over a fixed window, then promote from monitor to warn to block.
What is a circuit breaker for AI agents?
A circuit breaker automatically stops an AI agent when error rate, spend or tool failures cross a threshold, then tests recovery. States, triggers and thresholds.
Can you undo what an AI agent did?
Only some agent actions can be undone. Classify tools as reversible, compensable or irreversible, keep an action journal, and control irreversible tools up front.
How do you reconstruct what an AI agent did during an incident?
The trace data you need to rebuild an AI agent incident, why exact replay rarely works, and how to keep chain of custody for the evidence.
How do you stop an AI agent from emailing people outside the company?
Layer a recipient allowlist in the tool, mailbox-scoped send permissions, a mail flow or delivery rule, and alerts. A system prompt alone does not stop external email.
How do you set confidence thresholds for human review of AI document extraction?
Set review bands from measured accuracy, not raw model confidence. Sample auto-approved items, record each review, and recalibrate after model changes.
How do you stop a model update from silently changing agent behaviour?
Pin model IDs instead of aliases, track provider deprecation dates, and run regression evals before switching so a model update never changes an agent unseen.
What SLOs should you set for production AI agents?
Six SLOs for production AI agents: task success, groundedness, p95 latency, cost per task, escalation rate and policy violations, with how to measure each.
What is the lethal trifecta for AI agents?
The lethal trifecta: private data, untrusted content and external communication in one agent. Worked examples and controls that break each leg.
How do you stop an AI agent from deleting production data?
Seven layers that stop AI agents deleting production data: separate credentials, database privileges, deny rules, soft delete, backups and confirmation steps.
Do prompt injection detectors stop indirect prompt injection?
Detectors reduce indirect prompt injection but adaptive attacks get past them. Learn the design patterns that contain injection and how to test both.
What can you do to a misbehaving AI agent short of killing it?
A six-step response ladder for misbehaving AI agents, from alert to kill, with example triggers and who can authorise each step.
How do you regression-test an AI agent after changing its prompt?
Catch prompt-change regressions with a golden dataset, deterministic checks, calibrated LLM judges, fixed pass thresholds and CI release gates.
How do you detect data exfiltration by AI agents?
The signals that show an AI agent leaking data, from new destinations to encoded output and odd query volumes, plus the controls that stop it.
How do you govern an AI agent that extracts data from documents for underwriting?
Controls for underwriting extraction agents: classify first, per-field confidence thresholds, master-data checks and a reviewer approval record that holds up.
What is the difference between guardrails and evaluations for AI agents?
Guardrails block or change agent behaviour in real time; evaluations score it. Compare timing, cost, failure modes and the evidence each one produces.
How do you roll out and roll back a new AI agent version safely?
A staged canary plan for AI agent versions: traffic percentages, evaluation gates per stage, rollback triggers and what to record for each release.
What is excessive agency in AI agents?
Excessive agency (OWASP LLM06) is when an agent can do more than its task needs. The three root causes, examples and one control for each.
How do you use LLM judges to score production AI agent traffic?
Sampling rates, rubric design and calibration against human labels for LLM judges that score real AI agent conversations in production.
How do you stop memory poisoning in AI agents?
Memory poisoning plants instructions in an agent's long-term memory or context. Controls for writes, provenance, expiry and detection, mapped to OWASP ASI06.
What is policy as code for AI agents?
Policy as code for AI agents: rules written as versioned, tested files that decide what agents may call, send and spend, with examples and a review workflow.
How do you document which AI traffic paths your gateway actually controls?
Before you enforce AI policy, map every path from agent to model and tool. A traffic-path matrix, an acceptance test per path and common blind spots.
How do you govern what AI agents remember?
Treat agent memory as a governed asset: what to inventory, retention limits by memory type, how to honour erasure requests and who may read it.
What is a groundedness guardrail and when should it block a response?
How groundedness checks work, how they differ from task adherence, and which action (block, review, warn, log) to take for each type of AI agent answer.
How do you test that your AI agent kill switch actually works?
A 10-step game-day script to test an AI agent kill switch, the five timings to measure, pass criteria and the failures teams usually find.
How do you stop an AI agent from looping?
Stop AI agent loops with a max-step limit, a per-task token budget, repeated tool-call detection and a cost circuit breaker enforced outside the agent.
How do you control which websites and APIs an AI agent can reach?
Control AI agent network egress with per-agent allow-lists at an egress proxy, DNS policy and network identity, plus tool-level limits. A step-by-step guide.
How do you validate an AI agent's output before it acts?
Validate AI agent output in sequence: schema checks, business rules, groundedness and task adherence, then risk-based approval, before any tool call executes.
How do you tell users when an AI agent is shut down or quarantined?
Who to notify when an AI agent is quarantined or killed, what fallback to give users, a message template, and how to wire alerts to the quarantine event.
Should an LLM decide how severe an AI agent incident is?
LLM-assigned vs rule-assigned severity for AI agent incidents: where model triage helps, the rule-based floors to keep, and how to audit severity changes.
How do you detect behavioural drift in AI agents?
Detect AI agent behavioural drift by baselining tool-call mix, tokens, refusals and errors per agent version, then alerting on sustained deviation.
What should an AI agent incident response runbook include?
A phase-by-phase AI agent incident runbook: detect, contain, preserve evidence, investigate, recover and learn, with timings, roles and artefacts to keep.
How do you test AI agents with simulated users before production?
How to build a persona and scenario matrix, set pass thresholds with repeated runs, and how simulated-user testing differs from red-teaming AI agents.
Should AI agents do financial calculations themselves?
No. Let the agent plan and explain, and let deterministic code do the arithmetic. How to design calculation tools, thresholds and evidence for finance agents.
What should a production monitoring view for AI agents show?
A panel-by-panel spec for monitoring AI agents in production: status, severity-ranked issues, deviation, performance, quality, cost per agent and owner, posture.
How do you export AI agent traces with OpenTelemetry?
Export AI agent traces with OpenTelemetry GenAI conventions: span names, key gen_ai attributes, content capture rules and their Development status.
When should an AI agent be automatically quarantined?
A three-tier severity model for AI agents: which signals trigger automatic quarantine, which alert a human, which open a ticket, plus a pre-launch checklist.
How do you redact PII from LLM prompts at the gateway?
Redact, mask or block PII in LLM prompts at the gateway: a decision table by data type, layered detection and how to handle false positives.
How do you route AI agent policy violations into ITSM and incident tickets?
Route by severity: digest, owner ticket, on-call page, or automatic quarantine plus incident. Include agent, owner, policy and evidence in every ticket.
How do you enforce data residency for LLM and AI agent traffic?
Classify each request, route it only to model endpoints approved for that data class, and block or redact when no compliant route exists, at the gateway.
Which AI agent actions need human approval?
A scoring matrix for deciding which AI agent actions need human approval, by reversibility, value, data class and blast radius, with timeout rules.
How should you set severity levels for AI agent policies?
A severity rubric for AI agent policy findings based on data sensitivity, action reversibility and blast radius, with response times and example findings.
Put the framework into practice