Guardrail Fuzzer for typed decision models

I found no dedicated tool for this exact niche. The category itself is only weeks old: Cloudflare's Clef shipped on Workers AI around October 1, 2026, OpenAI's Decisions API reportedly went into public beta on October 6, 2026, and Kev is a community project. The paper that names the attack, "One Word Opens the Gate" (arXiv 2610.12292), is equally recent and comes with research code. So the attack taxonomy is published, but nobody has turned it into a reusable harness that takes a developer's own option schema and policy, runs against several backends, and reports fail-open and fail-closed rates plus the smallest attack that worked. General red-team tools like Promptfoo, garak, FuzzyAI and DeepTeam don't model typed options or definitions yet. A new entrant could win by shipping first with adapters for Clef, Kev, OpenAI decisions and Jev, a CI mode that fails builds when fail-open goes above a threshold, and built-in checks for the paper's defenses (dual labeling, passing only structured fields, neutral instructions). The main risks are these. Promptfoo, which may now belong to OpenAI, could add a decisions provider and option-channel plugins quickly. The paper authors could polish their repo into exactly this tool. And the paper's own finding that typed field parsing plus deterministic rules beats the model may push careful teams away from model-based gates, which would shrink the market. The best positioning is probably a free OSS CLI that becomes the standard benchmark for decision-model gates, rather than a paid product.

• option-channel-attack (research code for arXiv 2610.12292)

  This is the paper that defined the attack class gatefuzz would target: misleading option labels, definition smuggling, and log-text injection into the judged trace. It reports fail-open rates from 0% to 63% with injected log lines, and from 93% to 100% when only the permissive option is relabeled. It also proposes defenses, such as dual labeling and passing only structured fields. Search snippets say there is a code repo, but I could not confirm it has a runnable, general-purpose harness. It is the nearest direct predecessor and the source of the attack corpus, not a product.

• Kev benchmark suite (Jared Palmer)

  An open Jev-style decision model whose bundled benchmark checks accuracy, calibration, option-order sensitivity, question isolation and overconfidence on unanswerable items. These are robustness checks next to the attack, not adversarial fail-open fuzzing against a user's own policy.

• nicia-ai admission-decision-eval

  A one-off eval of Jev, Clef and Clef-flash on a knowledge-base write-admission gate. It found that Kev-4B was the only model to admit a write crafted to steer the classifier. It is a single-task benchmark, not a reusable fuzzer.

• Promptfoo (red team)

  A widely used open-source red-team tool with agent and tool-misuse plugins, reportedly acquired by OpenAI in 2026 (I could not confirm the acquisition from a primary source). It would be the obvious place to add a decisions-endpoint provider and option-channel plugins. Today it attacks generative LLM apps, not typed option/definition schemas.

• NVIDIA garak

  An open-source LLM vulnerability scanner with many probe modules and backends. It works at the model and prompt level and has no notion of decision-model option schemas or fail-open versus fail-closed gate metrics.

• CyberArk FuzzyAI

  An open-source (Apache 2.0) LLM jailbreak fuzzer with many attack techniques and support for custom attacks. It targets generative models, not typed allow/deny gates.

• Unit 42 AdvJudge-Zero

  An automated black-box fuzzer from Palo Alto Networks that attacks LLM-as-judge guardrails. It is conceptually the closest prior work for judge-style gates, but it is vendor research rather than an open harness for decision endpoints.

• DeepTeam

  An open-source red-teaming framework (Apache 2.0) covering agentic vulnerabilities and guardrails. It is a general LLM red-team tool with no decision-model or option-channel attacks.

• OrcaRouter firewall red-team recipe

  Vendor docs for testing a tool-call firewall: an eval harness over adversarial corpora, shadow mode, and precision/recall readouts. It shows that measuring tool-gate guardrails is a known practice, but it is tied to that vendor's own firewall.

Checked 2026-10-09