Advice Line: FCA disclaimer and advice-boundary testing for UK finance assistants

No vendor I found sells a ready-made, UK-specific regression suite that grades finance assistant conversations for disclaimers, drift from guidance into a personal recommendation, and mention of the complaints route and Financial Ombudsman, then outputs a dated Consumer Duty evidence pack. Demand looks timely. Around 19 to 20 October 2026, news reports said the FCA asked AI providers for clear, proportionate and consistent disclaimers and published good and poor practice notes from AI Live Testing that flag single-model and vendor dependence as a risk. Rerunning tests on every model change is the direct answer to that risk. The likely threats are Advai, which already provides the FCA's technical testing, and Aveni, which already sells Consumer Duty QA and a finance-specific model. Both could add this quickly, and the FCA itself might publish test sets. General AI testing platforms such as Giskard and Patronus have the engine but not the UK content. A new entrant would win on domain depth: a large, maintained set of conversations covering pensions, ISAs, debt, investing and the new targeted support regime, graded by people who know COBS, the FCA's conduct-of-business rules. It would also need an evidence pack shaped for board reporting. The retrieval sweep is weaker on its own, because generic knowledge-base poisoning scanners already exist. It works best as a feature of the main suite, aimed at the specific risk of text that suppresses regulator, complaint or risk-warning mentions.

• Advai

  UK automated AI testing and evaluation firm that is the FCA's technical partner in its AI Live Testing programme. Cohorts include financial advice, debt resolution and targeted investment support use cases at firms such as Barclays, Lloyds and UBS. It is the closest adjacent player, but it works through the regulator's programme rather than selling a productised regression suite to firms.

• Aveni (Detect and FinLLM)

  Edinburgh firm selling Aveni Detect, which reviews advice interactions for Consumer Duty and vulnerability and produces reports for the regulator. It also has FinLLM, a finance-specific model with an AveniBench evaluation set, now being tested at Nationwide. Well placed to extend into testing chatbot output.

• Giskard

  Automated red-teaming and evaluation platform for AI agents covering security and business compliance failures, with banking and insurance clients (BNP Paribas, AXA, BoursoBank). Generic; it has no UK advice/guidance boundary or Financial Ombudsman test content.

• Patronus AI

  Evaluation and simulation infrastructure for AI models and agents, used by financial services teams to check agent behaviour before and after deployment. Horizontal, not UK-regulation-specific.

• Saifr (Fidelity)

  Compliance tool from Fidelity that flags misleading text and missing disclosures in financial communications, and says it can sit on top of chatbot output. Built around FINRA and SEC rules rather than FCA ones.

• LatticeFlow AI (RAG knowledge base poisoning evaluation)

  Scores each knowledge-base document as clean or poisoned by embedded adversarial instructions. Overlaps with the retrieval sweep idea but is not tuned to suppressed regulator, complaint or risk-warning text.

• rag-inject-guard (open source)

  Open-source package that scans retrieved documents for instruction overrides, system-prompt manipulation and invisible Unicode, and quarantines flagged documents. Generic security tooling that covers the basic mechanics of the retrieval sweep.

Checked 2026-10-20