Refusal Map: which models still do legitimate defensive security work

Part of the problem is already being solved: the main labs now attack refusals directly. OpenAI's Trusted Access for Cyber and Anthropic's three-tier Cyber Verification Program (October 2026) both reduce friction for verified defenders, and Anthropic explicitly lets small firms and individual researchers apply. On the leaderboard side, Artificial Analysis appears to have launched a Cyber Index in September 2026 that reports refusal rates on defensive tasks. With its reach, a generic "which model refuses" chart is hard to win head-on. I found no maintained, open, weekly leaderboard limited to defender tasks such as YARA and Sigma rules, PoC explanation, log triage and CVE reachability. Nobody seems to report results per verification tier (unverified vs TAC vs CVP Defense vs Red Team), and nobody separates API content-filter blocks from model refusals. A practitioner-built open harness that covers exactly those points and is reproducible could become the neutral reference. That matters because the labs report little about over-refusal themselves: a Cloud Security Alliance guide notes most providers publish no over-refusal figures. I found no product for the router specifically. Existing gateways (Portkey, Bifrost, HydraLLM, Lunar) fall back only on HTTP status codes, not on detected refusals. But the moat is thin: refusal detection is easy to add to those gateways, and verification programs shrink the problem for anyone willing to get vetted. The best angle is probably to package it for small MSSPs and pentest shops as an audit trail. That means per-hop logs fit for client reports, routing to local open-weight or vetted-tier endpoints, and refusal-rate data from the leaderboard to decide routing order. The leaderboard then works as the distribution channel and the router as what customers pay for. Some figures here, including model names, refusal percentages and launch dates, came from secondary coverage I could not check against primary sources.

• Artificial Analysis Cyber Index

  This is the closest threat to the leaderboard idea. A secondary report says it is a public leaderboard launched in September 2026 that shows refusal rates alongside defensive code-security performance across frontier models, with incident response planned later. Artificial Analysis already has the brand and the traffic. I only saw it through secondary coverage, so the exact model names and numbers are unverified.

• Defensive Refusal Bias (Scale AI, ICLR 2026 workshop)

  An academic study of 2,390 defensive prompts taken from a collegiate cyber defense competition. Refusals were highest for system hardening (43.8%) and malware analysis (34.3%), and stating authorization made refusals more likely. It validates the problem, but it is a one-time paper, not a weekly tracker.

• defenders-dilemma

  A small single-author GitHub repo that measures refusals on incident-response artifacts across five models. It separates API-level content-filter blocks from model refusals. It is a proof of concept, not a maintained product.

• cyber-refusal-eval

  A hobby repo with a multi-turn eval of how cyber refusals decay over a conversation. It shows individuals are starting to build harnesses.

• CyberSecEval 2 (Meta)

  An established open benchmark from 2024 that includes a false refusal rate metric on borderline cyber prompts. It is dated and not run as a live leaderboard, but it is the reference people will compare against.

• OpenAI Trusted Access for Cyber / GPT-5.4-Cyber

  Big-lab feature that cuts the refusals themselves. Verified defenders get versions of the models with less safeguard friction, and higher tiers can use a permissive cyber-tuned variant. This weakens the router's value for customers who get verified.

• Anthropic Cyber Verification Program

  Expanded in October 2026 to Defense, Red Team and Specialized tiers. Small security firms and individual researchers are explicitly eligible, and the program cuts automated blocks on malware analysis and vulnerability work. This is a direct substitute for refusal routing among Anthropic users.

• WhiteRabbitNeo (Kindo)

  An uncensored open-weight security model built on Qwen. It is the kind of fallback target the router would send to, and it shows pentesters already get around refusals by self-hosting.

• Portkey / Bifrost / HydraLLM gateways

  Mature OpenAI-compatible gateways with fallback chains and logging, but they fall back on HTTP status codes, not on detected refusals. Adding refusal detection to them would take little work, so they are the likely fast followers for the router.

Checked 2026-10-15