Mosaic Check: test whether your knowledge agent pieces secrets together from allowed documents

I found no packaged open-source suite that ships seeded document sets where the secret only appears when documents are combined, and none that reports leak rates by scenario and query style for an internal knowledge agent. The academic work is recent and confirms the threat. MosaicLeaks targets deep-research agents leaking through web queries, and the RAG mosaic paper is a framing, not a tool. General red-team tools like Promptfoo, garak and PyRIT cover direct exfiltration and canary leaks. Knostic sells aggregation and oversharing controls to enterprises that buy Copilot or Glean, not testing to the startups that build the agents. A new entrant could win with a vendor-side, CI-friendly fixture library: realistic mid-market scenarios like a pending layoff inferred from headcount plans plus a budget memo, or an M&A target inferred from travel logs plus legal invoices. It would also need graded query styles and a shareable leak-rate report that startups can show buyers in security reviews. The main risks are that Promptfoo or a similar framework adds a mosaic plugin cheaply, that the open-source core is easy to copy, and that Seed and Series A teams may only pay when a customer's security questionnaire demands it. So the business likely depends on attestation reports, hosted runs or a certification-style badge rather than the fixtures alone.

• MosaicLeaks benchmark (arXiv 2605.30727)

  This May 2026 academic benchmark is the closest named match. It has 1,001 multi-hop deep-research tasks that chain private enterprise documents with a public web corpus, and it measures what an adversary can rebuild from the agent's outgoing web queries. It explicitly uses the mosaic framing, where single queries look harmless but reveal private facts together. Its threat model is leakage through external queries, not a knowledge agent assembling a secret in its answer to an internal user. It is research, not a packaged CI tool.

• Multi-turn mosaic and linkability attacks on RAG (arXiv 2603.17292)

  This paper uses a contextual-integrity framing for RAG and treats mosaic-style linkability as one of three attack vectors, alongside direct extraction and prompt injection. It is academic work that validates the threat. It is not a test suite that product teams can run against their own agents.

• Knostic

  Funded startup (about $19M raised) selling need-to-know controls for Copilot and Glean. It explicitly markets against LLMs combining content a user is allowed to see into sensitive conclusions it calls oversharing. It is aimed at enterprise buyers of Copilot and Glean, not at vendors building their own RAG agents, and it is a commercial runtime and monitoring product rather than an open test suite. It is still the most likely commercial competitor if it moves toward vendor-side testing.

• Promptfoo red teaming (RAG and agent plugins)

  Open-source red-teaming framework with YAML configs and plugins for RAG document exfiltration, cross-session leaks and RAG poisoning. It already has the format and distribution this idea wants, and someone could add a mosaic or aggregation plugin. I found no aggregation-inference scenario in it today.

• Kunal Ganglani RAG data leakage test suite tutorial

  A 2026 tutorial on a CI leakage suite that uses planted canary secrets in a staging knowledge base. It checks for direct leakage of a single canary, not secrets that only appear when documents are combined. It shows teams are building this themselves.

Checked 2026-10-24