Experts follow an AI agent's random lean 86% of the time, so advice firms will need influence audits

The influence audit looks open. I found no product that runs controlled framing swaps on adviser workflows and measures how far human recommendations shift. Evaluation vendors such as Vals AI, Patronus and ValidMind score what the AI outputs, not how much the professional using it changes their judgment. Automation-bias and anchoring work so far is academic, for example a 2026 computational pathology anchoring study and an AIES paper on the limits of reliance interventions. One arXiv governance paper proposes probe-based framing tests but nobody sells them. Demand is building because FINRA's 2026 oversight report gave generative AI its own section and treats the algorithm as part of the supervisory chain. Lawyers at Morrison & Foerster and Baker Donelson are telling RIAs and broker-dealers to do ongoing vendor diligence and audit AI usage. Still, no regulator has asked for measurements of human influence. A new entrant would have to create that category and probably sell it as evidence for supervision and Reg BI. I could not verify the 86% figure the pitch relies on, so the claim needs a real citation or the firm's own data. The live research-agent scorecard is more contested. Vals AI is well funded and already runs private finance evaluations. Patronus, ValidMind's vendor program, consultancies like CRISIL, and a growing set of free public benchmarks (FrontierFinance, Hedge-Bench, Finance Agent) all overlap. A newcomer could only win by being procurement-native: neutral and paid by the buyer, continuously re-running the specific agents a fund has licensed on its own live questions, and keeping an archive that can be checked in vendor reviews. Vals could add that quickly. Bundling both ideas into one audit-ready AI supervision record for compliance teams is the most defensible position.

• Vals AI

  An independent AI evaluation company. It publishes finance benchmarks, including the Finance Agent benchmark, and runs custom private evaluations on enterprise clients' own data and tasks. Andreessen Horowitz reportedly led a $40M Series A in August 2026. It is the strongest threat to the live research-agent scorecard idea. It scores model and agent outputs, not how much they move human judgment.

• Patronus AI

  An LLM evaluation and simulation platform that does hallucination detection, adversarial test generation and benchmarking. It built FinanceBench and sells into financial services. It is mostly aimed at development teams, not at procurement teams or compliance teams reviewing vendors.

• ValidMind

  AI model risk management for banks under SR 11-7 and the EU AI Act. Its Advantage program has third-party AI vendors test and document their models to banks' regulatory standards. It is a likely channel or competitor for the vendor-review part of either idea, but it focuses on model validation, not on human influence.

• CRISIL (vendor model validation)

  A services firm that validates vendor models for a large US institution's asset management and wealth units, using backtesting, benchmarking and ongoing monitoring. This is the incumbent consulting route that firms would use by default.

• Trata Hedge-Bench

  A YC startup's public benchmark built from hedge fund analysts' reasoning traces. It shows benchmark builders are moving toward buy-side research. It is a one-off public benchmark released by a vendor, not a continuous scorecard for a firm's own purchases.

• FrontierFinance / Deep FinResearch Bench (academic and JPMorgan)

  Open benchmarks for investment research agents. FrontierFinance plans to keep reporting results as new systems appear, and JPMorgan's Deep FinResearch Bench scores research reports. Both are free inputs that a scorecard product would build on, which also lowers what buyers will pay for generic scoring.

• Relyance AI third-party AI risk

  Continuous vendor AI posture checks using live telemetry. It covers privacy and compliance risk, not the accuracy of investment research or influence on advisers.

• WealthAi compliance tool

  Launched May 2026. AI agents review wealth manager communications in real time and generate audit records. It sits next to the compliance buyer but does not test how AI framing changes adviser recommendations.

Checked 2026-10-28