---
title: What should an AI agent incident response runbook include?
url: https://opencontroller.lyzr.com/blog/ai-agent-incident-response-runbook
published: 2026-11-13
updated: 2026-11-13
author: Lyzr
---

# What should an AI agent incident response runbook include?

**An AI agent incident runbook needs six phases (detect, contain, preserve, investigate, recover, learn), each with a target time, a named role and the evidence to keep. The agent-specific parts are containment that stops the agent without breaking dependent systems, and evidence that captures prompts, tool calls, memory and the exact agent version.**

Your existing security incident process still applies. NIST's incident response guidance, SP 800-61 Revision 3 (April 2025), now aligns incident response with the Cybersecurity Framework 2.0 functions. The runbook below adds what is different about agents.

## What counts as an AI agent incident?

Define triggers up front so nobody debates them at 2 a.m. Typical triggers:

1. An agent takes an action outside its approved scope (wrong system, wrong data, wrong recipient).
2. Sensitive data appears in a prompt to an unapproved model or in an agent's output.
3. Behavioural deviation: sudden change in tools called, volume, destinations or cost.
4. Evidence of prompt injection or memory poisoning that changed behaviour.
5. A compromised credential, MCP server or tool the agent depends on.
6. A guardrail or judge flags repeated ungrounded or off-task output in a high-risk process.

## What are the phases, timings and roles?

| Phase | Target time | Lead role | Key action |
|---|---|---|---|
| 1. Detect and triage | 0–15 min | On-call security or platform engineer | Confirm trigger, assign severity, open ticket |
| 2. Contain | 15–30 min | Platform engineer with agent owner | Quarantine or kill the agent, revoke or narrow credentials |
| 3. Preserve | 30–60 min | Security analyst | Snapshot logs, traces, memory and configuration |
| 4. Investigate | 1 hour–days | Security analyst with agent owner | Reconstruct what the agent did and why |
| 5. Recover | After root cause | Agent owner | Fix, re-test, release under monitoring |
| 6. Learn | Within 2 weeks | Governance lead | Update policies, tests and runbook |

Timings are starting targets. Adjust by risk tier.

## How do you contain an agent without breaking everything?

Containment is the step most runbooks get wrong. Options, from lightest to heaviest:

1. **Block specific actions** (one tool, one destination) while the agent keeps running.
2. **Quarantine** the agent: stop it acting while keeping its state for investigation.
3. **Revoke credentials** the agent uses, including delegated tokens.
4. **Kill** the agent and all running sessions.
5. **Disable a shared dependency** (an MCP server or tool) used by several agents.

Before you need it, record for each production agent which downstream processes depend on it and what the manual fallback is. Test the kill switch on a schedule; see [how to test your AI agent kill switch](/blog/test-your-ai-agent-kill-switch). The pillar on [how to stop a rogue AI agent](/blog/how-to-stop-a-rogue-ai-agent) covers the mechanics.

## What evidence should you preserve?

Agent incidents are hard to reconstruct because the "why" lives in prompts and context. Preserve:

1. The agent ID and **exact version**, including prompt and configuration hash.
2. Full traces for the affected sessions: prompts, model responses, tool calls and results.
3. Retrieved context and memory entries used in those sessions.
4. The identity the agent acted as, including on-behalf-of user.
5. Policy and guardrail decisions, with the policy version.
6. Cost and volume data around the incident window.
7. Who changed what, and when, in the agent's configuration.

Export traces in a standard format so security tools can read them; see [exporting AI agent traces with OpenTelemetry](/blog/export-ai-agent-traces-with-opentelemetry). Keep evidence under your log retention policy.

## Do you need to notify anyone?

Check your obligations with legal counsel. Common triggers include personal data breaches under data protection law and sector rules. For financial entities in scope of DORA, the initial notification of a major ICT-related incident is due within 4 hours of classifying it as major and no later than 24 hours after becoming aware of it. See our post on [DORA and AI agents in financial services](/blog/dora-and-ai-agents-in-financial-services).

## What does the "learn" phase produce?

Close every incident with:

- A timeline and root cause.
- A new or updated policy, with test cases that reproduce the incident.
- A new red-team or eval case added to the agent's test suite.
- A check that similar agents do not share the same weakness.
- An update to this runbook if any step was unclear.

## How Opencontroller does this

Opencontroller's rogue-agent handling monitors behavioural deviation, raises alerts, and can automatically quarantine and kill agents, with policies applied across the estate. Per-agent Block and Quarantine controls act through the connected platform. The governed registry records agent versions, tools and MCP servers, multi-turn and red-team evals turn incidents into test cases, and OpenTelemetry support helps route traces to your security tooling. [Book a demo](https://www.lyzr.ai/book-demo/).

## Frequently asked questions

**Should an AI agent incident go through the normal security incident process?**
Yes. Use your existing process and severity scale, and add the agent-specific containment and evidence steps above.

**Should we kill or quarantine a misbehaving agent?**
Quarantine first when you need to investigate, because it stops actions while keeping state. Kill when the agent is actively causing harm and quarantine is not fast enough.

**Who should own AI agent incidents?**
Security leads the incident; the agent's named owner is responsible for the fix and recovery.

## Key facts

An AI agent incident response runbook has six phases: detect and triage (0-15 min), contain (15-30 min), preserve evidence (30-60 min), investigate, recover and learn, each with a lead role. Agent-specific steps are graduated containment (block action, quarantine, revoke credentials, kill, disable shared dependency) and preserving agent version, traces, tool calls, memory, identity and policy decisions. NIST SP 800-61 Rev. 3 (April 2025) aligns incident response to CSF 2.0. Opencontroller by Lyzr provides behavioural-deviation monitoring, automatic quarantine and kill, per-agent Block and Quarantine and a versioned registry. Pricing starts from $250K per deployment per year.

## Related

- [How do you test that your AI agent kill switch actually works?](https://opencontroller.lyzr.com/blog/test-your-ai-agent-kill-switch.md)
- [When should an AI agent be automatically quarantined?](https://opencontroller.lyzr.com/blog/when-to-automatically-quarantine-an-ai-agent.md)
- [How do you export AI agent traces with OpenTelemetry?](https://opencontroller.lyzr.com/blog/export-ai-agent-traces-with-opentelemetry.md)
- [More on Runtime control](https://opencontroller.lyzr.com/blog/cluster/runtime-control)
