Policy Probe: multilingual policy audits for money-moving banking agents

This is a competitive space. Hamming already pitches policy-violation red teaming of bank agents with multilingual scenarios and CI. Giskard sells multi-turn agent testing to bank compliance teams. Generic simulation tools like Okareo, Coval and FutureAGI cover CI gating. So the CI gate idea is basically a commodity feature and won't stand alone. The narrower gap is the policy-to-test generator done as an audit artifact. That means taking a bank's written fee, confirmation and privacy rules and producing labelled trajectories tied to specific clauses. It also means translating them faithfully into the bank's actual customer languages, including US Spanish with code-switching, and reporting results by language: the same scenario passes in English but flips in Spanish, whether in the agent or in its guard. None of the vendors I found clearly ships clause-traceable policy coverage with differential results by language, packaged as evidence for model risk management (SR 11-7 style) and fair-lending or UDAAP reviewers. That buyer cares about language parity because of disparate-treatment exposure. A new entrant could win by owning that compliance-evidence framing and a curated banking policy and translation corpus, and by selling to model risk teams rather than engineers. The risk is that Hamming or Giskard add a "language parity report" as a feature, so the moat would have to come from domain-labelled data and regulator-ready reporting rather than from the test harness itself.

• Hamming AI

  Demoed at FinovateFall 2026. Red-teams bank voice and chat agents for policy violations, data leakage and authentication failures. Generates scenarios in multiple languages and regional variants (https://hamming.ai/product/automated-testing/scenario-generation) and markets CI/CD integration. This is the closest direct overlap with both product ideas.

• Giskard

  Evaluation and red-teaming platform sold to banking and insurance compliance teams. It generates test cases from a knowledge base and from agent descriptions, with a bank support agent as its own example. Hub 3.0 (Sept 2026) added multi-turn scenarios with checks on individual turns. It also published a red-teaming benchmark built on a banking assistant. I didn't confirm that it turns individual policy rules into labelled trajectories.

• Aveni Agent Approve

  Pre-production testing for financial-services agents. Runs synthetic personas and scores conversations against an FCA-aligned conduct and Consumer Duty taxonomy. In early access. Focused on the UK, and I saw no multilingual angle.

• Cekura

  Voice agent testing with explicit language-plus-accent pairing (for example, Spanish with a Latin American accent) and code-switching. Not specific to banking or policy.

• Enkrypt AI

  Agent red teaming that supports multilingual agents and maps results to NIST, OWASP and the EU AI Act. A general security tool rather than a fee or confirmation policy auditor for banks.

• Okareo / Coval / FutureAGI / Maxim (agent simulation tools)

  Generic agent simulation and regression tools with CI hooks, marketed to fintech. These cover the CI gate idea as general infrastructure, without bank policy content.

• Gradient Labs

  Builds AI agents for financial services and publishes guidance on staged evaluation involving compliance teams. Probably a builder that does its own evals, not a third-party auditor.

• Sierra tau3-bench banking domain / MAPS benchmark

  Open benchmarks. Tau3-bench has a banking domain with 97 tasks over 698 policy documents, in English only. MAPS (https://papers.lunadong.com/paper/7069) extends agent benchmarks into 11 languages and shows that non-English input causes wrong transfers in a banking example. Academic work like FraudBench and ReguSim also probes policy-grounded financial agents. Together these validate the problem and supply free building blocks.

Checked 2026-10-17