---
title: How do you check open-weight models before using them in agents?
url: https://opencontroller.lyzr.com/blog/scan-open-weight-models-before-use
published: 2026-11-15
updated: 2026-11-15
author: Lyzr
---

# How do you check open-weight models before using them in agents?

**Before an agent uses an open-weight model, check four things: the file format cannot run code when loaded, the source and exact revision are pinned, the licence allows your use, and the model behaves as expected on your own evaluations. Format checks stop the most common supply-chain attack. Only evaluation catches a model that is safe to load but wrong or tampered with.**

Downloading a model from a public hub feels like downloading data. It is closer to downloading software. This post covers the checks to run before a model reaches any agent. For running the approved model afterwards, see [governing open-weight models on-premises](/blog/govern-open-weight-models-on-premises).

## Why can loading a model run code?

Many older checkpoints use Python's pickle format, for example `pytorch_model.bin` or `.pt` files. Hugging Face's own documentation states that pickle lets "users to run python code at de-serialization time", and that arbitrary code execution attacks are possible when you load a pickle file. Loading is enough. You do not need to run inference.

Three facts matter here:

- **Hub scanning helps but is not a guarantee.** Hugging Face runs every upload through ClamAV and a pickle import scan that flags suspicious imports. Its documentation says this "is not 100% foolproof" and that checking safety is the user's responsibility.
- **Safetensors stores tensors only.** The format has no mechanism to execute code on load. Hugging Face published an external security audit of it in 2023 before making it the default.
- **PyTorch tightened its default.** From PyTorch 2.6, `torch.load` defaults to `weights_only=True`, which restricts what a pickle can reconstruct. Older code and pinned old versions may still load full pickles.

## What should the pre-use checklist include?

Run this for every model and every new revision.

1. **Prefer safetensors.** Reject pickle-only checkpoints unless there is no alternative. If you must convert one, do it in an isolated, network-less sandbox.
2. **Read the hub scan results.** Treat any flagged import as a hard stop.
3. **Run your own scanner.** Use an open-source model scanner in CI as a second opinion on every file in the repository, not only the weights.
4. **Disable remote code.** Keep `trust_remote_code=False` in Hugging Face Transformers. If a model needs custom code, review that code like any other dependency.
5. **Pin the exact revision.** Load by commit hash (`revision="<sha>"`), not by branch name. A branch can change after your review.
6. **Record file hashes.** Store SHA-256 hashes of every approved file and verify them at load time.
7. **Mirror internally.** Serve approved models from an internal registry or artifact store so production never pulls from the public hub.
8. **Check the publisher.** Prefer the original lab's organisation account. Watch for typo-squatted names and re-uploads with changed files.
9. **Read the licence.** Note use restrictions, attribution rules and thresholds. Some community licences require a separate agreement above a user count.
10. **Evaluate behaviour.** Run your task evaluations, a red-team set and safety tests before approval. Compare scores with the publisher's reported results.
11. **Register it.** Record owner, source, revision, hashes, licence, evaluation results and approved uses in your model inventory.

## Which risks does each check cover?

| Risk | Example | Check that catches it |
|---|---|---|
| Code execution on load | Malicious pickle payload | Safetensors only, scanners, `weights_only=True` |
| Malicious helper code | Custom modelling file that calls out | `trust_remote_code=False`, code review |
| Silent change after review | New commit on `main` | Revision pinning, hash verification |
| Impersonation | Look-alike organisation name | Publisher check, internal mirror |
| Licence breach | Commercial use where terms restrict it | Licence review |
| Backdoored or degraded weights | Trigger phrase changes outputs | Behavioural evaluation and red-teaming |
| Unknown models in use | Team pulls a model directly | Inventory and discovery |

Note the last two rows. A backdoored model in safetensors format passes every format check. Behavioural testing is the only pre-use control for that, and runtime monitoring is the control after.

## Where does this fit with agent governance?

A model check is one step in a wider supply chain that also includes MCP servers, skills and agent code. The OWASP Top 10 for Agentic Applications groups these under agentic supply-chain vulnerabilities (ASI04). The same pattern applies to each: review, pin, mirror internally, register and monitor. For skills, see [how to check agent skills for malicious instructions](/blog/scan-agent-skills-for-malicious-instructions). For tools, see the [MCP governance guide](/blog/how-to-govern-mcp-servers).

## How Opencontroller does this

Opencontroller auto-discovers agents, models and API keys across AWS, Azure and Google Cloud, so models deployed outside the approval process show up in the inventory. Multi-turn and red-team evals test a model in the context of the agents that use it. The LLM Gateway routes agent traffic to approved models with cost attribution per agent. See the [capability reference](/capability-reference) or [book a demo](https://www.lyzr.ai/book-demo/).

## Frequently asked questions

**Is a model safe if the hub scan shows no issues?**
No. Hugging Face says its scanning is not foolproof. Use safetensors, run your own scanner, and evaluate behaviour before approval.

**Are GGUF or ONNX files safer than pickle?**
They do not use pickle, which removes that specific code-execution path. Parsers for any format can still have bugs, so keep runtimes patched and load files in isolated environments.

**How do we stop teams pulling models straight from public hubs?**
Mirror approved models internally, block direct hub access from production networks, and use discovery to find models deployed outside the process.

## Key facts

Pickle-based checkpoints (.bin, .pt) can execute arbitrary Python code when loaded (Hugging Face docs). Hugging Face scans uploads with ClamAV and a pickle import scan but says it is not 100% foolproof. Safetensors stores tensors only and cannot execute code on load; Hugging Face published an external audit in 2023. PyTorch 2.6 changed torch.load default to weights_only=True. Pre-use checklist: prefer safetensors, own scanner, trust_remote_code=False, pin revision by commit hash, verify SHA-256, mirror internally, check publisher and licence, behavioural evaluation, register in inventory. Backdoored weights pass format checks; only evaluation and runtime monitoring catch them. Opencontroller (by Lyzr) auto-discovers models across AWS, Azure, Google Cloud and runs multi-turn and red-team evals. Pricing starts from $250K per deployment per year.

## Related

- [How do you govern open-weight models like DeepSeek or Qwen running on-premises?](https://opencontroller.lyzr.com/blog/govern-open-weight-models-on-premises.md)
- [How do you check agent skills for malicious instructions before use?](https://opencontroller.lyzr.com/blog/scan-agent-skills-for-malicious-instructions.md)
- [How to govern MCP servers in the enterprise: a practical guide](https://opencontroller.lyzr.com/blog/how-to-govern-mcp-servers.md)
- [More on Tools and MCP](https://opencontroller.lyzr.com/blog/cluster/tools-and-mcp)
