Rate Card: LLM discount benchmarks and switch tests during the lab price war

Each piece has an incumbent. Vendr, Vendor Benchmark and Tropic already pool and sell contract pricing data, and Braintrust and rightmodeler already replay production traces against cheaper models. ValidMind and Openlayer sell SR 11-7 validation documentation for LLMs. No one I found ties them together into a single renewal-ready report showing both that the buyer's discount is below what similar companies get and that a cheaper model passes a given share of their real prompts at a given cost per passed task. The benchmark firms have no technical proof, and the eval tools have no contract data and no finance buyer. The most defensible wedge is probably the regulated Switch File, because banks and fintech lenders need re-validation evidence before they can take a cheaper model, and current model-risk tools treat a model change as a general monitoring event rather than producing a ready-made memo for examiners. The weak spot is the pooled discount benchmark. It has a cold-start problem against Vendr's existing network, and lab prices are falling fast enough that discount data goes stale quickly. A standalone Switch Test is easy for Braintrust or the labs' own eval tooling to copy, so a new entrant should lead with the combined finance-plus-proof report or the regulated memo and treat the benchmark as an add-on rather than the core.

• Vendr (OpenAI / Anthropic pricing intelligence)

  Procurement platform that says it uses anonymised contract data to show what similar companies pay OpenAI and Anthropic and where buyers have leverage. This is a direct threat to the discount benchmark idea, but it has no technical switch evidence.

• Vendor Benchmark

  Gated benchmark that claims 140+ Anthropic deals, a 38% average API volume discount and a spread of up to 40% for comparable deployments. These are self-reported marketing figures, but the product overlaps the discount benchmark almost exactly.

• Tropic Price Benchmarks

  SKU-level price benchmarks across 14,000+ suppliers taken from live negotiations. I couldn't confirm LLM API coverage, but it could add it easily.

• Redress Compliance GenAI negotiation advisory

  Buyer-side advisory firm that claims 70+ OpenAI deals benchmarked and sells negotiation services for OpenAI and Anthropic contracts. Its published discount ranges are inconsistent, which suggests the underlying data is thin.

• SpendHound

  SaaS spend-management vendor that publishes guidance on negotiating generative AI pricing. It's an adjacent channel competitor for finance buyers.

• Braintrust

  Eval and observability platform that can run alternative models against saved production traces, score them side by side and attach cost to every call. It covers most of the Switch Test mechanics but isn't packaged for renewal negotiations or finance.

• rightmodeler

  Open-source CLI, listed August 2026, that replays agent traces through cheaper candidate models, measures agreement and savings, and packages approved swaps as PRs with evidence. Closest to the Switch Test, but I only found it on a directory listing, so it's unverified.

• Doubleword Migrate

  Migration offering for moving workloads off OpenAI and Anthropic to cheaper serving. It's an adjacent provider-led switch tool, not neutral.

• ValidMind

  Model risk management platform positioned for SR 11-7, SS1/23 and E-23, with LLM and foundation-model test support and validation report generation. It's the incumbent for regulated documentation, but I found nothing specific to model-swap re-validation memos.

• Openlayer

  AI eval and governance vendor marketing SR 11-7 (and a reported SR 26-2) validation for LLMs and agents, including revalidation triggers. A plausible competitor for the regulated Switch File.

Checked 2026-10-25