Leak Probe: cross-tenant cache and egress tests for hosted inference providers

I found no commercial product that runs continuous two-account canary and cache-timing tests against hosted inference providers and packages the results for bank and fintech third-party risk reviewers. The method itself is public and fairly mature: the Gu et al. ICML 2025 paper, the CacheProbe preprint, NDSS work on vLLM/SGLang KV-cache timing, and Promptfoo's open-source cross-session leak plugin all cover it. So the harness is not the moat. The opening is to be the independent, repeatable source of evidence. Demand looks real. Cloudflare's recent GLM-5.3 cross-tenant KV cache leak was caught by an outside researcher, not by continuous testing. A 2026 comment letter on the OCC's proposed third-party risk guidance complains that banks have no standard for AI vendor evidence and that every bank sends its own questionnaire. FS-ISAC and similar questionnaires ask questions but attach no empirical tests. Vendor Pack combined with Leak Probe CI is where a new entrant could win. The pitch is a standing, third-party-run isolation attestation, along the lines of a SOC 2 style addendum that is re-tested nightly. Mid-tier inference hosts like Together, Fireworks, DeepInfra, Baseten and the gateways would buy it to shorten bank procurement, and fintech risk teams would accept it in place of custom questionnaires. There are risks. The largest providers already run canary tests in-house and may not want an outsider publishing timing results. Black-box timing probes are noisy, and with only two accounts you can't prove isolation, only fail to find leaks, so the attestation needs careful wording. Pentest firms and red-team platforms like Promptfoo could add the feature cheaply. Egress Check is the weakest of the three ideas. Running open weights offline in a network-logged sandbox is a commodity that NVIDIA OpenShell, OpenSandbox, Azure and Vercel sandboxes already provide. Model weights themselves don't make network calls; serving code does. So this is better folded in as a small feature for model risk teams than sold as a standalone product.

• Auditing Prompt Caching in Language Model APIs (Gu et al., ICML 2025)

  Academic statistical timing audit. It found global prompt cache sharing across users at seven API providers, including OpenAI. This is the core method behind Leak Probe CI's timing probe, and it is published and reproducible by anyone, but it is not a product.

• CacheProbe: Auditing Prompt Cache Isolation in Gateway APIs

  May 2026 preprint that applies cache-timing audits to OpenRouter. It reports cache sharing under shared default credentials but not with direct API access or BYOK. More research showing the technique is getting known, but there is no commercial offering.

• Promptfoo Cross-Session Leak plugin

  Open-source red-team plugin. One simulated user plants data and a second session tries to pull it out. It works at the application level and doesn't use two real tenant accounts or timing analysis. Still, it is the closest packaged tool, and a team could adapt it.

• Cloudflare Workers AI GLM-5.3 cross-tenant leak disclosure

  Not a competitor. It shows the need is real: 155 responses leaked across tenants through a KV cache bug. Cloudflare checked its fix with the researcher's reproduction script and its own canary-marker tests, so big providers build this kind of testing in-house.

• RedFox Security multi-tenant AI leakage testing

  Pentest firm that publishes a two-tenant canary method for AI apps. This points to manual consulting engagements as the current alternative to automated nightly testing.

• Oxyne AI agent tenant isolation testing

  Security testing vendor writing about tenant isolation for AI agents in multi-tenant SaaS. It is aimed at app builders, not inference providers, and doesn't cover cache side channels.

• FS-ISAC Generative AI Vendor Risk Assessment workbook

  Free Excel questionnaire that banks use to tier due diligence on GenAI vendors. It is the default artifact Vendor Pack would have to plug into. It asks questions but includes no technical test evidence.

• Batoi community bank AI vendor assessment framework

  Weighted-scoring AI vendor questionnaire for banks. It covers the questionnaire side of Vendor Pack but not empirical isolation testing.

• Abrigo AI vendor questionnaire

  Bank-focused AI governance vendor checklist built for exam-ready documentation. Same limitation: questionnaire only.

• NVIDIA NemoClaw / OpenShell sandbox monitoring

  Live view of a sandbox's network connections and blocked egress. It is a general building block that could do what Egress Check proposes. Not aimed at model risk teams.

• Alibaba OpenSandbox egress sidecar

  Open-source FQDN egress allowlist for sandboxes. Another free building block for Egress Check.

Checked 2026-10-27