Fair Shake: matched-pair fairness testing for AI banking assistants
Automated test personas that probe a bank's AI assistant or adviser for unequal pricing, offers or advice by wealth, age or location, then produce an evidence report regulators will accept.
Competitive
Written for: Small startup team
Why now
A study found ChatGPT and Claude quote different shopping prices depending on how wealthy the user seems. Backbase made its agentic conversational banking generally available, HSBC is cutting up to 70% of UK advisers in favour of AI, and MAS now requires independent review before AI goes live. Banks are shipping AI that talks to customers before they have a way to show it treats customers equally.
What became possible
Fair-lending regulators have long sent matched human testers who differ only in one trait. That method can now be automated cheaply. LLMs generate thousands of personas that are identical except for one attribute. Decision models such as OpenAI's Decisions API ($0.10 per million input tokens) or Liquid's d1 then score each conversation for price, product offered and how suitable the advice is, without needing output tokens.
Two ways in
- Paired-persona test harness
Connects to a bank's chat or voice assistant in staging. It runs pairs of personas that differ only by a protected or proxy attribute (postcode, income signals, name, age), records quotes, offers and refusals, and runs significance tests on the gaps.
- Customer
- Head of Conduct Risk at a UK retail bank rolling out an AI assistant or robo-adviser under Consumer Duty
- Model
- $40-80k a year per assistant tested, plus a per-release regression run
- Vendor certification report
A repeatable fairness test report that conversational-banking vendors such as Backbase give to their bank customers to help close procurement.
- Customer
- Conversational-banking and wealth-tech vendors selling to regulated banks
- Model
- Fee per certification plus an annual retest subscription
Competition
No product found that is specialised for UK banking and Consumer Duty and produces statistically tested matched-pair evidence on wealth or income, postcode, age and vulnerability. Existing tools mostly vary race and gender, or are generic red-teaming and security tools.
Full competition notes (7 competitors)
No product found that is specialised for UK banking and Consumer Duty and produces statistically tested matched-pair evidence on wealth or income, postcode, age and vulnerability. Existing tools mostly vary race and gender, or are generic red-teaming and security tools. A new entrant could win by testing those attributes and delivering an evidence pack mapped to Consumer Duty outcomes, with significance-tested gaps in quotes, offers and refusals. No fairness certification report for conversational-banking vendors (such as Backbase) to give bank buyers was found, so that angle looks open. Risks: FairPlay or Giskard could add this as a feature quickly, Advai's FCA position could make it the default supplier, and the FCA's good and poor practice report on AI (late 2026) could favour incumbents or create a standard to build to. The defensible edge is deep UK and Consumer Duty specialisation, not the test harness.
- FairNow – Conversational AI & Chatbot Bias Assessment
Listed in the UK government's AI assurance catalogue. Fills realistic synthetic prompts with race or gender signals (the same counterfactual-persona method). Doesn't appear to cover wealth, postcode or age, or anything banking-specific.
- FairPlay AI
Fair-lending testing firm backed by JPMorgan Chase. Now markets AI and agent testing, validation and red-teaming for banks, including pre-launch guardrail checks. US-focused, aligned with SR 11-7 (US model-risk guidance), and its fairness work centres on credit models.
- Giskard
Automated red-teaming for AI agents. Tests BPCE's customer-facing banking assistants continuously, and its scans include bias and stereotype probes. Emphasis is security and compliance, not statistical fairness evidence.
- Advai (FCA AI Live Testing technical partner)
The FCA's technical partner for AI Live Testing, already working with NatWest, Monzo, Santander, Barclays and Lloyds. Strongly placed to own 'evidence the regulator accepts'.
- Holistic AI
Tools for LLM bias benchmarking and adversarial testing. Audited Starling Bank, but that covered face verification during onboarding, not a chatbot.
- SolasAI
Disparity testing and bias reduction for credit and insurance decision models. No conversational AI offering found.
- Crowdtesting for bank AI systems (banking.vision)
German provider that sells bias testing of bank chatbots by age, gender and origin using human testers.
Risks
- Choosing protected-attribute proxies is legally sensitive and needs advice from compliance counsel
- LLM outputs vary from run to run, so detecting gaps needs large samples and careful statistics
- Large model-risk consultancies may bundle this into their existing services
Strong team fit; market pull and open space are the weak point (7).
tech 8 · pull 7 · space 7 · fit 9 · build 8
Exposed
Companies that could benefit or be threatened. News and context, not investment advice.
Momentum of its main topic
Sources
- Study: Claude, ChatGPT Offer Different Shopping Prices Based on WealthHacker News
- Backbase puts agentic AI in customers' handsFinextra
- HSBC to make 'deep, wide and brutal' UK wealth management job cuts in AI pushFinextra
- Singapore central bank issues risk rules on banks' AI useFinextra
- OpenAI launches Decisions API that reduces complex evaluations to yes, no, or pick oneThe Decoder
- LiquidAI/d1-3B · Hugging Facer/LocalLLaMA
What happened next
- +7d · 15 Oct 2026 nothing yet. No launches, funding rounds, acquisitions, or significant press coverage of Fair Shake or matched-pair fairness testing for AI banking assistants were found after the October 8, 2026 flag date. The most recent relevant development was 360factors' August 2026 launch of a fair lending testing agent, which predates the flag.
Status history
- 2026-10-08 Flagged as buildable