---
title: "Lyzr Gateway Benchmark Report"
description: "Benchmark report highlights and PDF download."
canonical: https://opencontroller.lyzr.com/llm-gateway-benchmark
last-updated: 2026-10-26
---

# Lyzr Gateway Benchmark Report

Engineering research / October 2026

# Lyzr Gateway Benchmark Report

Measured performance. Practical trade-offs.
Routing, model cost, caching and payload limits, tested by Lyzr AI engineering.

[Download Report](/__l5e/assets-v1/8a7a3653-bd77-487e-948d-b7f2e3ac1e99/lyzr-llm-gateway-benchmark-report-2026-10-25.pdf)PDF · 10 pages · Version 1.0

<0.7 ms

Added p95 routing latency

In-gateway methods; mock upstream

5,134

Requests per second

Single 4-vCPU gateway; no rules

23.5%

Lower model cost with caching

Mixed synthetic traffic stream

90 MB

Requests carried successfully

Local payload test; mock upstream

Inside the report

## Four questions. One measured baseline.

01

### What does routing add?

Every tested in-gateway routing method added less than 0.7 ms at p95. The external classifier added about 143 ms at p95.

02

### Where can model cost come down?

A cheaper model at low thinking effort matched a premium model at high effort at 47% lower list price on 540 paired, machine-gradable questions, not open-ended tasks.

03

### When does caching pay off?

The mixed synthetic stream reduced model cost by 23.5%, with zero incorrect cached answers in the tested configuration.

04

### How large can payloads be?

Local tests carried 90 MB requests and 50 MB responses at 100% success. Provider limits still apply; Anthropic’s tested limit was 64 MiB.

![Cover of Lyzr’s Performance and Cost Benchmark Report, published 25 October 2026](/__l5e/assets-v1/a8b1afba-747e-4124-9801-711c02819923/gateway-benchmark-cover.jpg)

The complete report · 25 October 2026

Caching / mixed traffic

## Savings that depend on the workload.

For 1,000 interleaved requests across six synthetic applications, model cost fell from $12.02 to $9.20, a 23.5% reduction, with a 19.9% cache hit rate.

Repetitive single-workload tests showed 91–98% reductions. Those are not typical-traffic forecasts. Real savings need validation on your own workloads.

Model cost per 1,000 requests / USD

Without cache$12.02

With cache$9.20

Claude Opus 5.5 · provider list prices · synthetic mixed stream. Source: report §4.2.

A finding worth keeping in view

## Record-specific traffic needs exact matching.

Semantic matching can confuse prompts that differ only by a claim number, name or amount. The report recommends exact-only caching, or no caching, for record-specific traffic. The full report includes correctness checks and the recommended configuration.

Methodology & scope

## Evidence, not a promise.

Tests ran from 7 August to 21 October 2026 using public benchmark sets and synthetic workloads only. No customer data was used.

2,031,800 requests in the routing campaign; mock upstream isolates gateway overhead.

MMLU and GSM8K plus generated questions; open-ended and non-English quality not evaluated.

21,400 caching requests; savings depend on repetition and configuration.

Single-gateway and local payload results; not multi-replica or production-traffic guarantees.

## Go beyond the highlights.

Get every table, test condition and configuration recommendation.

[Download Report](/__l5e/assets-v1/8a7a3653-bd77-487e-948d-b7f2e3ac1e99/lyzr-llm-gateway-benchmark-report-2026-10-25.pdf)[Book a demo](https://www.lyzr.ai/book-demo/)

## Your privacy choices

With your permission, we and our service providers use cookies and similar technologies to recognize returning visitors, measure how the site is used, and support marketing. Essential website functions remain available if you decline. [Read our privacy policy.](/privacy)
