---
title: How do you stop a model update from silently changing agent behaviour?
url: https://opencontroller.lyzr.com/blog/pin-model-versions-for-ai-agents
published: 2026-11-15
updated: 2026-11-15
author: Lyzr
---

# How do you stop a model update from silently changing agent behaviour?

**Pin every agent to a fixed model ID rather than an alias that can move, record that ID as part of the agent's version, and only switch models after regression evals pass. Then track each provider's deprecation schedule so retirements become planned upgrades, not outages. Finally, monitor behaviour in production, because providers note that infrastructure changes can shift outputs slightly even when the model ID stays the same.**

A common incident report reads: "the agent started doing something different and nobody changed anything." Often someone did change something: the model behind an alias. This post shows how to make model changes deliberate. For the broader release process, see [canary releases and rollback for AI agents](/blog/canary-releases-and-rollback-for-ai-agents).

## What is the difference between a model alias and a snapshot?

| | Alias | Pinned model ID (snapshot) |
|---|---|---|
| Example | `claude-sonnet-4-5`, `gpt-4o` | `claude-sonnet-4-5-20250929`, `gpt-4o-2024-08-06` |
| Can the underlying model change? | Yes, it can point to a newer snapshot | No, fixed for the life of the ID |
| Good for | Prototypes, quick tests | Production agents |
| Risk | Silent behaviour change | Retirement date you must track |

Anthropic's [model IDs documentation](https://platform.claude.com/docs/en/about-claude/models/model-ids-and-versions) says that for earlier models, short aliases point to the most recent dated snapshot. From the 4.6 generation, dateless IDs such as `claude-sonnet-4-6` are themselves fixed snapshots, and updated versions ship under new IDs. Anthropic states that the underlying model stays constant for the lifetime of a model ID, a guarantee that covers IDs, not convenience aliases.

## How much notice do providers give before retiring a model?

Retirement turns a pinned model into an error. Anthropic's [deprecation policy](https://platform.claude.com/docs/en/about-claude/model-deprecations) gives at least 60 days' notice before retiring a publicly released model. For example, `claude-opus-4-1-20250805` was deprecated on 5 June 2026 and retired on 5 August 2026.

Two details catch teams out:

- **Cloud platforms set their own dates.** Anthropic notes that Amazon Bedrock and Google Cloud set their own schedules, so the same model can retire on different dates depending on where you call it.
- **Sixty days is short for regulated change control.** If your release process needs approvals and testing, start when a model is deprecated, not when retirement is close.

## How do you pin and upgrade models safely?

1. **Inventory model usage.** List every agent, the model ID it calls and the platform it calls through. Flag any alias.
2. **Pin in configuration, not code.** Store the model ID in the agent's versioned configuration so changes are reviewed.
3. **Make the model part of the agent version.** Agent version = model ID + prompt + tools + policy. Change any one and you have a new version.
4. **Track deprecations.** Subscribe to provider notices and check each pinned ID against deprecation pages weekly.
5. **Keep a regression suite.** Maintain a fixed set of representative tasks, including multi-turn and adversarial cases, with pass thresholds.
6. **Run evals before switching.** Compare the candidate model against the current one on task success, groundedness, cost per task and latency. See [test a cheaper model before switching an agent](/blog/test-a-cheaper-model-before-switching-an-agent).
7. **Roll out gradually.** Send a small share of traffic to the new version, compare live quality scores, then expand.
8. **Keep a fallback.** Configure gateway fallback to the previous model if the new one errors.

## Why can behaviour still change with a pinned model?

Anthropic notes that infrastructure updates can produce minor differences in observable behaviour even when the model ID and weights have not changed. Other drift sources include changed tool outputs, new data in retrieval indexes and different user inputs.

So pinning is necessary but not enough. Monitor production quality continuously and compare against the baseline for each version. [Detect behavioural drift in AI agents](/blog/detect-behavioural-drift-in-ai-agents) covers the signals to watch.

## What should a model change record contain?

| Field | Example |
|---|---|
| Agent and new version | Claims triage v14 |
| Old and new model IDs | Pinned IDs, not aliases |
| Reason | Deprecation, cost, quality |
| Eval results | Pass rates vs thresholds, old vs new |
| Rollout plan | Canary share, duration, success criteria |
| Approver | Named owner |
| Rollback | Previous version ID and trigger |

## How Opencontroller does this

Opencontroller's governed registry tracks agents and their versions, and auto-discovery finds the models and API keys in use across AWS, Azure and Google Cloud, which helps surface aliases. Multi-turn evals test candidate versions, and LLM judges score production traffic per version. The LLM Gateway handles routing, fallback and retry, and attributes cost per agent and per version, so a model switch shows up in both quality and spend. See the pillar on [how to stop a rogue AI agent](/blog/how-to-stop-a-rogue-ai-agent) or [book a demo](https://www.lyzr.ai/book-demo/).

## Frequently asked questions

**Should production agents use model aliases?**
No. Aliases can point to a newer model without notice. Pin a fixed model ID and change it deliberately.

**How much notice does Anthropic give before retiring a model?**
At least 60 days for publicly released models. Amazon Bedrock and Google Cloud set their own schedules.

**Does pinning a model guarantee identical behaviour?**
No. Weights stay fixed, but infrastructure updates, tool outputs and retrieved data can still change behaviour, so keep monitoring.

## Key facts

Pin AI agents to fixed model IDs, not aliases. Anthropic: model ID guarantees constant underlying model; pre-4.6 aliases point to most recent dated snapshot; 4.6+ dateless IDs are snapshots; at least 60 days notice before retiring public models; Bedrock and Google Cloud set own schedules; claude-opus-4-1-20250805 deprecated 5 June 2026, retired 5 August 2026. Infrastructure updates can cause minor behaviour differences. Opencontroller: governed registry of agents and versions, auto-discovery of models and keys, multi-turn evals, LLM judges, LLM Gateway fallback and per-version cost attribution. Pricing starts from $250K per deployment per year.

## Related

- [How do you test whether a cheaper model can replace the one an AI agent uses?](https://opencontroller.lyzr.com/blog/test-a-cheaper-model-before-switching-an-agent.md)
- [How do you detect behavioural drift in AI agents?](https://opencontroller.lyzr.com/blog/detect-behavioural-drift-in-ai-agents.md)
- [How do you roll out and roll back a new AI agent version safely?](https://opencontroller.lyzr.com/blog/canary-releases-and-rollback-for-ai-agents.md)
- [More on Runtime control](https://opencontroller.lyzr.com/blog/cluster/runtime-control)
