Fault Line: replay realistic MCP failures against your agent before users hit them
Open-source MCP proxy that injects timeouts, errors and bad payloads in CI. Agent teams pay for hosted runs.
Competitive · Scale doesn't buy fault tolerance
Written for: Solo builder
Next step
This week, write an MCP proxy that injects timeouts, 500s, truncated JSON and stale results on a seeded schedule and publish it on npm with a GitHub Action. Then post a before-and-after run of a coding agent against a flaky filesystem server on r/LocalLLaMA and in the MCP community Discord.
Why now
Most teams assume a stronger model will ride out a flaky tool. The paper says it won't. With OpenAI offering discounts of up to 90% to win Anthropic customers, teams are switching models more often, and every switch needs a resilience retest. Our earlier Tool Fuzz cards covered security fuzzing of MCP servers. This card is the other side: does the agent recover when a server is slow, wrong or down? Nobody ships a simple CI tool for that yet.
What became possible
The MCP-Injector paper adds a fault-injection layer that sits in the MCP tool-call path. It can make faults permanent, transient or intermittent, uses seedable schedules, and matches its fault mix to real incident data. The finding: bigger models are not more fault-tolerant. Two harness papers (VCR and Model-Harness Fit, plus TraceTrial-Bench) supply the metrics, such as how often an agent still finishes normally when tools misbehave.
Three ways in
- fault-line CLI
A drop-in stdio/HTTP MCP proxy with seeded fault schedules: timeouts, 5xx errors, truncated JSON, stale results, schema drift. It reports completion rate and wrong-answer rate per tool.
- Customer
- Developers building on Claude Code, Codex or their own agent loop with MCP servers
- Model
- Free under MIT; a paid hosted runner keeps run history
- Resilience Action
A GitHub Action that replays a fixed task set under the same fault seed on every PR or model switch and fails the build if completion rate drops.
- Customer
- Small teams shipping agent features
- Model
- $29–99/month per repo for hosted runs and trend charts
- Model resilience table
A public leaderboard: the same tasks and the same fault seeds across open and closed models, rerun after each price cut.
- Customer
- Teams choosing a model during the price war
- Model
- Free leaderboard that drives sign-ups for the paid Action
Competition
The open-source proxy and the CI gate are already covered by mcp-drill, AgentBreak and MCP Failure Lab, so Fault Line would not be first with that pitch. The existing tools focus on whether an agent survives timeouts and malformed payloads.
Full competition notes (11 competitors)
The open-source proxy and the CI gate are already covered by mcp-drill, AgentBreak and MCP Failure Lab, so Fault Line would not be first with that pitch. The existing tools focus on whether an agent survives timeouts and malformed payloads. A new entrant could still win by measuring wrong-answer rate on stale or subtly corrupted results, which needs a judge or ground truth for each task and is where AgentCheck shows agents actually fail. It could also win with a true stdio proxy that runs real Claude Code or Codex agent loops against a fixed task set with seeded schedules, since current Action examples run mostly in mock mode. The clearest opening is a neutral public model resilience leaderboard, rerun as models and prices change. Today that exists only as one-off academic benchmarks such as ToolMisuseBench, ReliabilityBench and ToolBench-X, and it would be the cheapest way to get attention for the hosted product. The risk is that the hosted runs are easy to copy: the incumbent open-source tools could add these features quickly, and eval platforms like Braintrust or vendors like Harness could fold fault injection into products teams already pay for.
- mcp-drill
The most direct overlap. It wraps an MCP server and injects timeouts, malformed JSON-RPC and corrupted-but-valid tool outputs. It runs in CI without a live model and its listing says it also ships as a GitHub Action.
- AgentBreak
A chaos proxy between an agent and its LLM or MCP backend that you enable by changing the base URL. It injects latency, HTTP errors and malformed responses and serves a scorecard. Its docs show a GitHub Actions example that fails the build below a score threshold, though that example runs in mock mode with no agent in the loop.
- MCP Failure Lab
An npm toolkit for chaos and resilience testing of MCP servers over stdio and Streamable HTTP, covering delay, hang, disconnect and malformed messages with CI-friendly reporting. Its listing says it is not a general-purpose proxy.
- MCP Chaos Monkey
A chaos-injection project for MCP described on Medium, now with Python support. A smaller project in the same niche.
- Harness Resilience Testing MCP
A chaos-engineering vendor with an MCP feature in its resilience-testing product. It shows large incumbents are moving toward this space.
- Gravitee AI gateway
An API and agent gateway vendor publishing a lot of content on fault injection for MCP and LLM traffic, including schema drift, timeouts and 429 storms, positioned around its own gateway.
- Braintrust MCP evals
An eval platform that recommends measuring task completion with error-path cases on every merge or nightly. It could add fault injection to an offering teams already pay for.
- ToolMisuseBench
An offline deterministic benchmark from April 2026 with replayable faults (schema drift, rate limits, timeouts, auth failures) and 6,800 tasks. It overlaps the leaderboard idea but is an academic one-off.
- ReliabilityBench
A benchmark that measures agent fault tolerance under timeouts, rate limits, partial responses and schema drift across models.
- ToolBench-X
A benchmark of tool-using agents under unreliable tool environments that injects hazards such as execution failure, output drift and cross-source conflict. Its listing page appears to have a leaderboard, but I could not confirm its rankings.
- AgentCheck
A research workbench that replays recorded MCP agent runs with one of 12 tool-response fault types. It found most failures were silent acceptance of bad or stale output rather than crashes.
Checked 2026-10-26
Risks
- The paper's authors may open-source MCP-Injector, which would make the proxy a commodity; then the value is the Action and the task sets
- LangSmith, Braintrust or promptfoo could add fault injection as a feature
- Many teams don't run agent evals at all, so willingness to pay is unproven
Strong team fit; market pull is the weak point (5).
tech 8 · pull 5 · space 6 · fit 9 · build 8
Exposed
Companies that could benefit or be threatened. News and context, not investment advice.
Momentum of its main topic
Sources
- Tool-Calling LLM Agents Under Realistic MCP Faults: Scale Does Not Purchase Fault TolerancearXiv
- When Agents Fail: Model or Harness--Who's to Blame?arXiv
- Trial by Trace: Benchmarking AI Agent Harnesses for Reliability and Failure AttributionarXiv
- OpenAI offers discounts up to 90 percent to poach Anthropic's biggest customersThe Decoder
Status history
- 2026-10-26 Flagged as buildable