Replay Leak Scan: catch third-party scripts reading your bank chatbot's conversations
Scripted chats with planted fake account details show which analytics and session-replay scripts capture assistant conversations. Fintech privacy counsel pay.
Competitive · First measurement of chatbot conversation leakage
Written for: Small startup team
Next step
Call the chief privacy officer at a digital bank that has launched a web AI assistant, offer a free canary scan of their logged-out chat, and in the same week ask a privacy-litigation defence partner which clients are already being served demand letters.
Why now
Banks and fintechs are adding AI assistants to web and app pages that already run session replay and analytics tags. The paper shows those tags can capture what customers type to the assistant, including account details. Wiretap-style privacy suits over session replay already cost companies money, and financial data adds GLBA exposure. The test itself is mechanical: send chats containing planted fake numbers and watch where they go.
What became possible
'Penny for Your Prompts' is the first systematic measurement of how third-party analytics, ad and session-replay scripts running next to AI chat interfaces quietly receive conversation data. It confirms this leak path is common.
Two ways in
- Canary Chat Crawl
A headless browser logs in with a test account, sends scripted chats seeded with fake card numbers, SSNs and account numbers, and records every third-party request that carries them. The output is a report with request evidence.
- Customer
- Privacy and legal teams at digital banks, credit unions and lenders with web chat assistants
- Model
- One-off assessment fee, then monthly monitoring per domain
- Release Watch
Rerun the canary chats on every front-end deploy and alert when a new tag starts receiving conversation text.
- Customer
- Front-end platform teams at fintechs
- Model
- SaaS subscription priced per monitored app
Competition
Third-party script monitoring and finding data leaks to trackers is an established category. Reflectiz, Feroot, Lokker and c/side all sell it, some directly to banks and framed around CIPA chat and session-replay wiretap risk.
Full competition notes (7 competitors)
Third-party script monitoring and finding data leaks to trackers is an established category. Reflectiz, Feroot, Lokker and c/side all sell it, some directly to banks and framed around CIPA chat and session-replay wiretap risk. Feroot already uses synthetic users. What I did not find is any product that logs into a bank's authenticated area with a test account, holds a scripted multi-turn conversation with an AI chat assistant seeded with canary card numbers, SSNs and account numbers, and produces request-by-request evidence of which third parties received the conversation text. That evidence is what counsel want in wiretap and GLBA disputes. Academic papers have already proved the method works on consumer chatbots, so the remaining value is in making it a product and packaging it for counsel. Tying the canary runs to front-end deploys (Release Watch) is also not offered as a product, although incumbents alert on new scripts in general. The risk is that this is a thin feature that Feroot or Reflectiz could add quickly to their existing synthetic crawlers. Each customer also needs to issue test credentials for logged-in testing, which adds sales friction. A new entrant wins only by being the specialist in chat-assistant canary testing, with litigation-grade reports and a quick fit into deploy pipelines. Partnering with or being acquired by an incumbent is a realistic outcome.
- Reflectiz
Agentless continuous monitoring of third-party scripts and pixels; published research showing bank sites sending loan amounts and tax IDs to TikTok, Google and Meta, and a case of a TikTok pixel reading a bank-style login form. Sells privacy compliance to financial services.
- Feroot (Inspector / PageGuard)
Uses synthetic users to crawl sites and map which scripts transfer what data; offers a free privacy assessment and explicitly advises checking whether chat widgets read form fields and make external requests. Strong in PCI DSS 6.4.3/11.6.1 script monitoring.
- Lokker
Privacy scanning sold around CIPA wiretap risk, explicitly flagging session replay, chat tools and form capture scripts that send inputs to third parties.
- c/side
Client-side third-party script monitoring that observes what data each script reads and where it sends it; has a scanner that categorizes scripts including session replay.
- Blacklight (The Markup)
Free, open-source headless-browser privacy inspector that detects session recording and key logging; public pages only, no logged-in or chat flows.
- Tracking Conversations (UC Davis research)
Academic measurement showing chatbots sending conversation text in plaintext to Microsoft Clarity session replay; demonstrates the method but is not a product.
- Penny for Your Prompts (research audit)
2026 audit of 42 AI chat platforms finding 9 leak verbatim prompts to third-party trackers; validates the canary-chat method academically, not commercialized.
Checked 2026-10-28
Risks
- Client-side script security vendors already sell PCI page monitoring and could add chat canaries quickly
- Testing behind login needs a test account from each customer, which slows sales
- The buyer may simply turn off session replay on chat pages once told, so the product needs ongoing monitoring to retain them
Strong buildability; market pull and open space are the weak point (6).
tech 8 · pull 6 · space 6 · fit 7 · build 9
Exposed
Companies that could benefit or be threatened. News and context, not investment advice.
Momentum of its main topic
Sources
Status history
- 2026-10-28 Flagged as buildable