Proving AI compliance verdicts actually follow the rule

I found no product that runs rule-deletion, swap and negation tests on AML or KYC AI dispositions and packages the results as evidence a second-line validator can file. The pieces are spread out. ValidMind, Openlayer and Fairplay offer generic SR 11-7 validation and documentation. Giskard does general LLM testing. AML vendors such as Axle and Elliptic validate their own models. The method itself exists in 2026 arXiv papers. A new entrant could win by owning AML/KYC-specific test suites built around real regulatory rules and typologies, by being the independent third party examiners prefer over vendor self-attestation, and by producing documentation formatted for model-risk files and for independent reviewers working for banks under enforcement pressure. It could also sell to the outside validators banks already rely on, or plug into the generic platforms. There are two main risks. First, several readings of the April 2026 interagency guidance (SR 26-2) put generative and agentic AI formally outside model-risk scope, which could weaken the compliance pull. Second, the tests are simple enough for Giskard, Openlayer or ValidMind to add as a feature, so any lasting edge has to come from domain-specific test content and from acceptance by examiners and independent reviewers. MAS guidance and the vendors Citadel AI, Yields.io and Chatterbox Labs were not checked in this review.

• ValidMind

  Model risk management and AI governance platform for regulated financial institutions, focused on testing and documenting models against SR 11-7. Generic validation tooling with no AML rule-sensitivity tests that I could find, but it is the obvious place model-risk teams already work.

• Openlayer

  Markets SR 11-7 / SR 26-2 validation for LLMs and agents, including outcome analysis and monitoring. Its evaluation framework is general purpose and could absorb rule-deletion tests.

• Fairplay.ai

  Sells automated SR 11-7 validation with exam-ready documentation, bias testing and agent evaluations. Its roots are in fairness and credit, and I found no AML rule-sensitivity product.

• Giskard

  LLM agent red-teaming and quality testing (hallucination, contradiction, omission) that claims BNP Paribas, Societe Generale and BPCE as customers. Its open-source library also has input-perturbation tests (my own knowledge, not from the searches), so a rule-sensitivity suite would be a small extension.

• Axle

  AML AI vendor that publishes guidance on getting AI agents through bank model validation (end-to-end workflow validation, challenger models). This is first-party validation support, not independent rule-sensitivity evidence.

• Elliptic

  Discusses counterfactual evaluation and A/B testing of crypto AML monitoring rules and models, but for its own detection logic rather than as an LLM-verdict audit product.

• What Do Compliance Detectors Read? (arXiv 2608.16852)

  Academic audit, not a product, that runs these delete, swap and reorder tests on guard models. Deleting the violated rule flipped verdicts only 44.5% of the time and swapping in a permissive rule only 7.1%, even though the model cited the right rule 91-95% of the time. Figures are from search summaries, not the full paper. It is the methodological basis for the idea and is public.

• Explainable AML Triage with LLMs (arXiv 2604.19755)

  Research proposing decision counterfactuals and evidence-removal counterfactuals as a validation layer for LLM AML triage. It shows the technique is known in the AML context but not productized.

Checked 2026-10-09