Eval containment: sealed offline worlds and egress-audited sandboxes

Generic website replicas are already competitive. Chakra Labs, Plato and Halluminate sell cloned sites and SaaS sandboxes. Microsoft (Echoverse, Fara-1.5) and academic projects like VeriEnv and CUA-Gym can auto-clone a site cheaply, so the replicas alone won't stay defensible. Those players aim at RL training throughput on commerce, productivity and enterprise SaaS apps. I found none offering curated, versioned replicas of high-risk civic and financial sites (government benefit forms, banking portals, police tip lines) built for safety evaluation, not training. I also found none with canary detection that flags when an agent tries to reach or act on the real version of a site. That vertical, with provenance and versioning that a safety institute could cite in a report, looks open, but the buyer pool is small: frontier labs and a handful of national AI safety institutes. The egress guard is earlier but no longer empty. Straiker launched Heimdall in September 2026 on exactly the eval-escape story. GKE Agent Sandbox and general sandbox vendors cover isolation, and AISI builds its own hardened setups. What still looks unclaimed is independent, tamper-evident proof of non-reach that a lab can hand to a regulator, something like an attestation or audit product rather than another sandbox. Winning there probably depends on becoming the neutral standard that safety institutes or a regulation adopt, not on beating sandbox vendors at isolation.

• Chakra Labs (Dojo)

  A startup selling RL environments for computer-use agents. A third-party listing says it offers deterministic clones of production software and has raised about $10M. That listing is the only source I have for the figures, and I haven't verified them. It focuses on training rather than safety evals of high-risk site types.

• Plato (plato.so)

  Builds simulated copies of sites like Amazon, Airbnb and Gmail as hosted environments for browser agents, with state tracking, scoring, an API and a Python SDK. It is small and early, with no disclosed funding or customers.

• Halluminate (YC)

  Sells managed, parallel sandboxes modeled on Salesforce, Slack, ticketing tools and websites, plus benchmarks and datasets. It explicitly pitches them as safer than testing agents on live sites. Its focus is enterprise SaaS, not government or banking forms.

• VeriEnv (ICML 2026)

  An academic framework in which a coding agent automatically clones a target website, including frontend, backend and database, into an executable sandbox. This makes custom replicas much cheaper to build.

• Microsoft Echoverse / Fara-1.5

  Microsoft's open-source environment factory for computer-use agents. The related Fara-1.5 paper builds sandboxed site clones from recorded human trajectories. This is a big lab doing replica generation in-house and giving it away.

• CUA-Gym

  A research suite of 94 mock web apps with session state you can inspect, inject into and reset. It argues real websites are unsuitable because they can't be reset or reproduced.

• Straiker Heimdall

  Launched September 2026 by Straiker, which raised a $64M Series A. It is a hardened sandbox for high-risk agent evals and training, with separate isolation, egress, telemetry and shutdown controls. It is pitched directly in response to the Hugging Face incident where an eval escaped into production. This is the closest match to the egress-guard idea.

• UK AISI SandboxEscapeBench

  The UK AI Security Institute runs agent sandboxes inside hardened VMs. It has also found that a sandboxed agent can work out who the operator is from DNS, certificates and proxy behavior. So the safety institutes are building this kind of containment themselves. My source for this is a third-party blog summary, not the AISI publication itself.

• Singapore AI Agents Sandbox (CSA/GovTech/IMDA with Google)

  A government sandbox that tested computer-use agents on government websites and social-assistance applications. One trial seeded inactive pages and staging-URL mismatches for agents to catch. This shows governments are interested in testing agents on civic forms.

Checked 2026-10-10