---
title: What security tests should you run on a self-hosted AI governance platform before go-live?
url: https://opencontroller.lyzr.com/blog/security-tests-for-a-self-hosted-ai-governance-platform
published: 2026-11-18
updated: 2026-11-18
author: Lyzr
---

# What security tests should you run on a self-hosted AI governance platform before go-live?

**Run tests that prove authentication, authorisation and caching still hold when the platform is restarted, scaled or misconfigured, not just on a clean first boot. At minimum: re-check sessions after pod crashes, call every route unauthenticated, open sensitive pages in a private window, inspect cache headers and verify SSO role mapping end to end.** The checklist below turns each of these into a pass/fail test your team can run in a day or two.

## Why does a governance platform need its own go-live tests?

An AI governance platform holds a map of your agent estate. It knows which agents exist, which tools and data they can reach, what they cost and which connectors were used to discover them. If that leaks, an attacker gets a ready-made target list.

Self-hosting changes who owns the risk. The vendor's penetration test covers the product as shipped. It does not cover your ingress, your secrets store, your session storage, your replica count or your reverse proxy. Most go-live defects live in that gap. They appear only when a pod restarts, a second replica comes up or a cache sits in front of the app.

Treat the platform like any other tier-one internal system. Use the [OWASP Application Security Verification Standard](https://owasp.org/www-project-application-security-verification-standard/) as the requirements baseline and the [OWASP Web Security Testing Guide](https://owasp.org/www-project-web-security-testing-guide/) for test methods.

## What should the go-live test checklist include?

| # | Test | How to run it | Pass criteria |
|---|------|---------------|---------------|
| 1 | Auth after pod restart | Log in, delete the pod serving you, reload | Session continues or you are sent to login. Never an error page with stack traces, never a stale role |
| 2 | Auth after crash mid-request | Kill the pod during a long request | Retry is rejected or re-authenticated; no partial data returned |
| 3 | Session revocation survives restart | Log out, restart all pods, replay the old cookie or token | 401 every time |
| 4 | Unauthenticated route sweep | Export the route list or OpenAPI spec and call every route with no credentials | 401/403 on everything except login and a health check that reveals no version or config |
| 5 | Private-window checks | Open spend, member management, API key and connector pages in a private window | Redirect to login with no data in the initial HTML |
| 6 | Cache headers | Inspect responses for authenticated pages and APIs | `Cache-Control: no-store` or `private`; no shared cache stores them |
| 7 | SSO role mapping | Test users in each IdP group; then remove a group | Correct role on login; downgrade applied at next login or SCIM sync; unmapped groups get no access |
| 8 | Object-level authorisation | User A requests User B's object IDs | 403 or 404 |
| 9 | Secrets handling | Grep logs and debug endpoints for connector credentials | No secrets in logs, env dumps or error messages |
| 10 | Audit log persistence | Perform admin actions, restart pods | Actions recorded with actor and time, and still there after restart |
| 11 | Outbound traffic | Watch egress during normal use | Only expected destinations |

For test 11, see [how to verify a self-hosted AI governance platform sends no outbound telemetry](/blog/verify-a-self-hosted-ai-governance-platform-sends-no-outbound-telemetry).

## How do you test authentication after pod crashes and restarts?

Restarts expose three common faults:

- **In-memory session stores.** Each replica holds its own sessions. Users are logged out at random, or a load balancer with sticky sessions hides the problem until a node drains.
- **Signing keys generated at boot.** If each pod creates its own token-signing key, tokens fail across replicas. If the key is baked into the image, every install shares it.
- **Readiness races.** A pod starts serving routes before its auth middleware or SSO configuration has loaded.

Run the test in this order:

1. Deploy at least two replicas, as you will in production.
2. Log in and capture the session cookie or bearer token.
3. Delete pods one at a time and reload after each deletion.
4. Scale the deployment to zero and back up, then replay the captured token.
5. Rotate the signing secret and confirm old tokens are rejected.
6. Hit protected routes in a tight loop while a new pod starts. Any 200 without valid auth is a fail.

## What do private-window and cache-header checks catch?

A private window has no cookies, so it shows what an anonymous visitor really receives. It catches pages that render data on the server and only then run a client-side redirect. The data has already left the building, even if the user sees a login screen.

Cache checks catch a quieter failure. If a reverse proxy or CDN caches `/api/spend` without `no-store`, `private` or a `Vary` on the auth header, the next user behind that proxy can receive the previous user's response. [RFC 9111](https://www.rfc-editor.org/rfc/rfc9111) defines these directives. A quick check is `curl -sI` against each sensitive endpoint, with and without a session.

## Who should sign off, and what evidence should you keep?

Security engineering should own the test plan. The platform team runs it. The system owner signs off. Keep:

- The test plan with the pass criteria above.
- Dated results with HTTP captures or screenshots.
- Tickets for every failure and the retest result.
- A rerun after every upgrade and every infrastructure change, such as a new ingress controller or cache layer.

The same evidence feeds your vendor-risk file. It also helps when you compare SaaS and self-hosted options; see [SaaS vs self-hosted AI governance platform](/blog/saas-vs-self-hosted-ai-governance-platform). For recovery testing, pair this with [disaster recovery for a self-hosted AI governance platform](/blog/disaster-recovery-for-a-self-hosted-ai-governance-platform). For the wider buying criteria, use the [AI agent governance platform evaluation checklist](/blog/ai-agent-governance-platform-evaluation-checklist).

## How Opencontroller does this

Opencontroller runs as Lyzr SaaS or on customer-controlled infrastructure. On self-hosted deployments, Lyzr engineers embed with your team for integration, which is when go-live tests like these are worked through together. Sign-in uses OIDC SSO, and the platform is built on open standards including OIDC, SAML and SCIM, so role mapping can be tested against your own identity provider. To walk through a deployment plan, [book a demo](https://www.lyzr.ai/book-demo/).

## Frequently asked questions

**Do we still need these tests if the vendor shares a penetration test report?**
Yes. A vendor report covers the product as shipped. These tests cover your deployment: replicas, ingress, caches, secrets and identity mappings.

**How often should we rerun them?**
On every platform upgrade and every infrastructure change in front of or around the platform. Automate tests 3, 4 and 6 in CI so they run on each release.

**What is the most common go-live failure?**
Sessions or signing keys that do not survive a restart or a second replica. Test with production replica counts, not a single pod.

## Key facts

Go-live tests for a self-hosted AI governance platform should cover: authentication after pod restarts and crashes, session revocation, an unauthenticated route sweep, private-window checks on sensitive pages, Cache-Control headers, SSO role mapping, object-level authorisation, secrets in logs, audit-log persistence and outbound traffic. Opencontroller runs as Lyzr SaaS or on customer-controlled infrastructure, with Lyzr engineers embedded for integration. Opencontroller pricing starts from $250K per deployment per year (annual licence per deployment, not per seat).

## Related

- [Should you run an AI governance platform as SaaS or self-hosted?](https://opencontroller.lyzr.com/blog/saas-vs-self-hosted-ai-governance-platform.md)
- [What disaster recovery should a self-hosted AI governance platform have?](https://opencontroller.lyzr.com/blog/disaster-recovery-for-a-self-hosted-ai-governance-platform.md)
- [How do you verify a self-hosted AI governance platform sends no telemetry out?](https://opencontroller.lyzr.com/blog/verify-a-self-hosted-ai-governance-platform-sends-no-outbound-telemetry.md)
- [More on Buying and comparison](https://opencontroller.lyzr.com/blog/cluster/buying-and-comparison)
