Haiku-priced E2E testing for indie web apps
Plain-English end-to-end tests run by a cheap computer-use model against your staging site on every deploy, which removes most of the work of writing Playwright tests.
Crowded
Written for: Solo builder
Next step
This week, build a GitHub Action that reads a tests.md file, drives a headless browser with Haiku 5.5 computer use against the preview URL, and comments the results; then list it on the Vercel integrations marketplace and post a cost-per-run breakdown on Indie Hackers.
Why now
Before this release, computer-use QA was either too unreliable or too expensive to run on every commit for a small app. Browser infrastructure like Kernel is now available off the shelf.
What became possible
Haiku 5.5 jumped from 15.7% to 72.4% on OSWorld computer use while its token prices fell by up to 90%, so a screen-reading test agent now costs cents per run instead of dollars.
One way in
- plaintest
Write tests like 'sign up with a new email, upgrade to Pro, cancel'. A Haiku 5.5 agent runs them in a hosted browser on each Vercel or Netlify preview and posts pass/fail with screenshots and a replay to the PR.
- Customer
- Solo founders and small SaaS teams on Vercel or Netlify with no QA staff
- Model
- Self-serve, $29 to $149 a month by number of runs
Competition
Plain-English tests that run on every preview deploy and post results to the PR no longer stand out on their own. TesterArmy, QA.tech, TestSprite and Momentic all do it, and Shortest offers it free if you bring your own API key.
Full competition notes (8 competitors)
Plain-English tests that run on every preview deploy and post results to the PR no longer stand out on their own. TesterArmy, QA.tech, TestSprite and Momentic all do it, and Shortest offers it free if you bring your own API key. Using a cheap Haiku-class model mainly cuts your costs, which rivals can copy by switching models. The realistic openings are narrower. First, a flat price with unlimited runs instead of unpredictable credit billing, which Bug0 (a competitor) already criticizes in TestSprite. Second, saving the agent's first successful run as a fixed script and replaying it, using the AI only when something breaks, which answers the non-determinism complaint. TesterArmy's own docs suggest pairing its agent with fixed regression tests. Third, handling the hard indie flows out of the box: getting past Vercel and Netlify deployment protection, real email signup, and Stripe test-mode upgrades and cancellations. Fourth, a cheaper hosted, no-setup version of Shortest to convert open-source users, though that is a thin advantage. Without one of these, a new entrant would mostly be competing on distribution, for example a Vercel Marketplace listing.
- TesterArmy
Almost exactly the plaintest idea. On every PR its agent reads the change, writes a test plan and runs it in a real browser. It hears about Vercel previews automatically through GitHub deployment events and updates a single PR comment on each push. It can also run saved regression tests alongside.
- QA.tech
Described as turning plain-English descriptions into E2E tests that run on every PR, pointed at the Vercel preview URL. Marketed as part of the Linear, Cursor and Vercel stack, so it targets the same buyers.
- TestSprite
AI testing agent already sold at solo-founder prices: a free tier, Starter at $19 a month and Standard at $69 a month, billed in credits. Its blog positions Starter for solo developers and small teams.
- Shortest (antiwork)
Free open-source framework, about 5.6k GitHub stars by one listing. You write tests in plain English and Claude runs them through Playwright, with support for GitHub 2FA and Mailosaur email checks. You bring your own Anthropic API key and run it in your own CI. Latest release seen was v0.4.9 from April 2025.
- Momentic
Funded platform where an AI agent writes and maintains natural-language tests and runs them in CI, reporting to GitHub, Slack and Linear. Pitched as a replacement for brittle Playwright suites. One comparison gives a starting price of $49 a month.
- Octomind
Berlin startup aimed at early-stage SaaS teams with no QA staff. Generates Playwright tests from plain English, and the tests fix themselves when the UI changes. Paid plans reportedly start around $146 a month.
- TestMu AI Kane CLI
Takes a Vercel preview URL and a plain-English goal and needs no Playwright code. TestMu is the former LambdaTest, so a large testing company is in this niche too.
- Bug0
Sells AI-driven E2E testing and publishes comparison pages against TestSprite and others, which shows competitors are already fighting over the same buyers.
Checked 2026-10-09
Risks
- Crowded field (QA Wolf, Momentic, Playwright plus AI codegen)
- 72% on OSWorld still means flaky runs, which need retries and caching
- Frontier labs or Vercel could bundle this
Strong team fit and buildability; open space is the weak point (4).
tech 7 · pull 6 · space 4 · fit 8 · build 8
Exposed
Companies that could benefit or be threatened. News and context, not investment advice.
Momentum of its main topic
Sources
What happened next
- +7d · 16 Oct 2026 nothing yet. No product launch, funding announcement, or significant press coverage specifically matching the Haiku-priced E2E testing opportunity was found between October 9 and October 16, 2026. Several plain-English E2E testing tools exist in the market, but none confirmed a launch or major milestone in this specific week.
Status history
- 2026-10-09 Flagged as buildable