---
title: Should an LLM decide how severe an AI agent incident is?
url: https://opencontroller.lyzr.com/blog/who-decides-severity-of-ai-agent-incidents
published: 2026-11-13
updated: 2026-11-13
author: Lyzr
---

# Should an LLM decide how severe an AI agent incident is?

**Not on its own. Let rules set a severity floor from facts you can check, such as data class, production status and whether the agent took an external action. Let an LLM add context, group duplicates and suggest raising severity, but never lower it below the rule-based floor without a person signing off. Log every severity change with who or what made it.**

The question comes up in nearly every security review of an agent governance tool: who decides whether an agent issue is critical, high or medium? Is it detected automatically, or configured? The honest answer is both, and the split matters.

## What do rule-based and LLM-based severity each do well?

| | Rule-assigned severity | LLM-assigned severity |
|---|---|---|
| Input | Structured facts: policy ID, data class, environment, action type | Free text, traces, prompts, outputs, past incidents |
| Consistency | Same facts always give the same answer | Can vary between runs and model versions |
| Explainability | The rule is the explanation | Needs a written rationale, which may be wrong |
| Coverage | Only what someone wrote a rule for | Can recognise new patterns |
| Speed to change | Policy change and review | Prompt change, which is easy to make silently |
| Audit | Strong | Needs logging of model, prompt and output |
| Gaming risk | Low | Prompt injection in the evidence can influence the verdict |

The last row is the one teams forget. If an LLM reads agent outputs to judge severity, and those outputs contain attacker-controlled text, the attacker can try to talk the triage model down. The OWASP guidance on prompt injection applies to your security tooling as much as to the agent.

## Which severity rules should never be overridden by a model?

Write these as hard floors. A model can raise severity above them, never below.

1. **Regulated or restricted data left its boundary.** Personal data, payment data, health data or secrets in an output, a tool call or an external request: at least High.
2. **Production agent took an external action that was not allowed.** Sent an email, moved money, changed a record, called an unapproved API: at least High.
3. **Agent acted outside its declared tools or scope.** Called a tool not on its allow-list: at least High.
4. **Customer-facing agent produced unsafe or prohibited content.** At least High.
5. **Credential or key exposure.** At least Critical until the credential is rotated.
6. **Repeat within a window.** The same policy breach by the same agent version twice in 24 hours: raise one level.
7. **Non-production, no external action, no sensitive data.** Low, unless another rule fires.

These rules use facts the platform already knows: the agent's environment and risk tier from the registry, the data class from classifiers or redaction, and the action from the tool call log. See [how to set severity levels for AI agent policies](/blog/how-to-set-severity-levels-for-ai-agent-policies) for building the policy side.

## Where does LLM triage add real value?

Models are useful in the parts of triage that are reading-heavy and low-stakes to get slightly wrong.

- **Summarising the incident** from a long trace into three sentences a responder can read.
- **Grouping duplicates.** Fifty alerts from the same root cause become one incident.
- **Spotting what rules missed.** "This output contains what looks like an internal project roadmap" when no classifier matched.
- **Suggesting an upgrade.** "Severity Medium by rule, but the trace shows the agent tried three times to reach an external file-sharing domain."
- **Drafting the timeline** for the post-incident review.

In each case the model proposes and a rule or a person decides. That keeps the speed benefit without making severity depend on a prompt.

## How do you audit severity decisions?

Severity drives response times, escalation and sometimes regulatory reporting. Treat changes to it as auditable events.

Log for every incident:

- Initial severity, the rule that set it, and the timestamp.
- Any LLM suggestion: model, model version, prompt version, the suggestion and its stated rationale.
- Every change: old value, new value, who or what changed it, and why.
- Any downgrade below the rule floor, with the approver.

Review a sample monthly. Look for patterns such as one team consistently downgrading, or a model suggesting downgrades for one kind of agent. NIST SP 800-61 Rev. 3 places incident prioritisation inside the organisation's wider risk management, which is a good argument for keeping the floors in policy, owned by the risk function, rather than in a triage prompt owned by whoever last edited it.

## How should severity connect to the response?

Severity is only useful if it changes what happens next. A simple mapping:

| Severity | Automatic action | Notify | Response target |
|---|---|---|---|
| Critical | Quarantine agent, rotate credentials | On-call, owner, CISO delegate | Minutes |
| High | Block the action, alert, open ticket | Owner, security queue | Same business day |
| Medium | Alert and ticket | Owner | Within a week |
| Low | Log for trend review | None | Monthly review |

Route tickets into the tools responders already use; see [route AI agent policy violations to ITSM](/blog/route-ai-agent-policy-violations-to-itsm). The end-to-end playbook is in [how to stop a rogue AI agent](/blog/how-to-stop-a-rogue-ai-agent) and the [AI agent incident response runbook](/blog/ai-agent-incident-response-runbook).

## How Opencontroller does this

Opencontroller applies policy as code across the agent estate, with built-in mappings to OWASP, MITRE ATLAS, NIST and ISO 42001, so severity floors live in reviewable policy. LLM judges score production traffic, live groundedness and task-adherence guardrails flag problems in the path, and behavioural-deviation monitoring can raise alerts and automatically quarantine or kill an agent. Posture detects risky permissions and configurations. See the [capability reference](/capability-reference) or [book a demo](https://www.lyzr.ai/book-demo/).

## Frequently asked questions

**Can an LLM lower the severity of an AI agent incident?**
It can suggest a downgrade, but it should not apply one below the rule-based floor without a person approving it and the change being logged.

**What facts should set AI agent incident severity?**
Data class involved, production vs non-production, whether an external action was taken, whether the agent acted outside its declared tools, credential exposure, and repetition.

**Can prompt injection affect LLM-based incident triage?**
Yes. If the triage model reads attacker-controlled agent outputs, that text can try to influence its verdict. Rule-based floors limit the damage.

## Key facts

AI agent incident severity should not be decided by an LLM alone. Rules set a severity floor from checkable facts (restricted data leaving its boundary, unauthorised external actions by production agents, tools outside the allow-list, unsafe customer-facing content, credential exposure, repeats); an LLM may summarise, group duplicates, spot what rules missed and suggest upgrades, but downgrades below the floor need human approval. Log the model, prompt version, suggestion and every severity change. Prompt injection in evidence can influence LLM triage. Opencontroller by Lyzr provides policy as code with mappings to OWASP, MITRE ATLAS, NIST and ISO 42001, LLM judges on production traffic, live guardrails, and automatic quarantine and kill.

## Related

- [How should you set severity levels for AI agent policies?](https://opencontroller.lyzr.com/blog/how-to-set-severity-levels-for-ai-agent-policies.md)
- [How do you route AI agent policy violations into ITSM and incident tickets?](https://opencontroller.lyzr.com/blog/route-ai-agent-policy-violations-to-itsm.md)
- [What should an AI agent incident response runbook include?](https://opencontroller.lyzr.com/blog/ai-agent-incident-response-runbook.md)
- [More on Runtime control](https://opencontroller.lyzr.com/blog/cluster/runtime-control)
