How it works
Follow one real run, artifact by artifact.
Our whole pitch is verifiable proof — so this page shows you the proof. Everything below is pulled straight from a real, already-merged run on an open-source app, not a mock-up. You never have to trust the agent. You only have to check the artifact it leaves behind.
$ npx @omea/cli run
✓ booted the app in an isolated VM
✓ authored a candidate against an uncovered route
✓ kept it — coverage moved (31 strictly-new lines)
evidence: .omea/last-run/coverage.mdWhy an artifact, not a checkmark
The agent that wrote the change doesn't get to grade it.
"Claude will often just update the tests to no-op so tests passing isn't trustworthy." A green checkmark from the same agent that wrote the code is a self-report. An artifact you can re-open, re-run, and re-hash is not.
So instead of asking you to believe a run happened, Omea hands you the things a run produces — a kept Playwright test, the coverage lines it newly touched, a sealed evidence bundle, and a weekly report where every number points back to one of those files. Walk the chain below; each piece is from the same real run.
Step 1 — the app boots itself
Your repo boots in an isolated microVM, zero config.
A read-only GitHub App connects the repo — you pick which ones, we never push. In the run shown here, boot-engine started vercel/next.js's examples/blog-starter in a KVM microVM from a clean node_24-bookworm snapshot and served it inside the guest, with dependencies and environment wired automatically. Nothing to stand up on your side, and each candidate test runs in its own forked VM, so one test can never leak state into another.
Step 2 — the tests you keep
An agent writes tests, but a test is only kept when it covers new lines.
Coverage is measured by watching the app actually run, never by static analysis. The agent picks an uncovered route, writes a plain @playwright/test file, and the run keeps it only if it moved coverage. This is the exact spec kept from our own run on the Next.js blog-starter example — stock-runnable with npx playwright test, no proprietary runner:
const { test, expect } = require("@playwright/test");
test("omea blog starter visits post and returns home", async ({ page }) => {
await page.goto(process.env.OMEA_APP_URL || "/");
await expect(page.getByRole("heading", { name: /Blog\./ })).toBeVisible();
await page.getByRole("link", { name: /Dynamic Routing and Static Generation/ }).first().click();
await expect(page).toHaveURL(/dynamic-routing/);
await expect(page.getByRole("heading", { name: /Dynamic Routing and Static Generation/ })).toBeVisible();
await expect(page.getByText(/Lorem ipsum dolor sit amet/).first()).toBeVisible();
await page.getByRole("link", { name: "Blog", exact: true }).click();
await expect(page.getByRole("heading", { name: /Blog\./ })).toBeVisible();
});It was kept because it added 31 strictly-new covered lines across 5 files — including the posts/[slug] page chunk and the post document, exactly the surface a "visit a post" test should newly cover. That is not our summary of the run; it is the run's own coverage-delta.json:
{
"schema_version": 1,
"strictly_new_covered_lines": 31,
"files": {
".../chunks/node_modules_next_dist_1ybzpk2._.js": [2706, 2710, 2731, 2735, 3100, 3101, 3102, 3103, 3104],
".../chunks/node_modules_next_dist_client_0r5nbpw._.js": [8018, 8019, 13421, 13422],
".../react-server-dom-turbopack_164kp-6._.js": [1409, 1410, 2438, 2439, 2440, 2441, 2491, 2492, 2493, 2494, 2498, 2499],
".../chunks/src_app_posts_%5Bslug%5D_page_tsx_1pwrm9a._.js": [1, 2, 3, 4, 5],
"http://127.0.0.1:3000/posts/dynamic-routing": [1]
},
"before_summary": { "coveredLines": 12952, "fileCount": 25, "testCount": 1 },
"after_summary": { "coveredLines": 12983, "fileCount": 27, "testCount": 1 },
"full_files_sha256": {
"coverage-before.json": "1a17ee23...",
"coverage-after.json": "218d7936..."
}
}A candidate that moves no new line is discarded, not kept. "Coverage went up" here means specific, listed lines — you can open the file and count them.
Step 3 — the evidence bundle
Every run seals into one file you can verify offline.
omea report packages a completed run into a single replayable artifact, evidence-<run-id>.tar.zst. It is just a directory you can open:
manifest.json version, repo, commit, tool versions, a SHA-256 for
every payload file, and an honest missing[] list
verdicts.ndjson the full run output stream, copied byte-for-byte,
never re-serialized
coverage/ raw before/after coverage JSON, plus the delta.json above
kept-tests/ the kept Playwright specs, exactly as recorded
artifacts/ traces, screenshots, videos when the run produced them
README.md offline instructions to verify the bundle and replayThen anyone can check it, with us nowhere in the loop:
$ omea report evidence-<run-id>.tar.zstThe integrity idea in one sentence: a SHA-256 is a fingerprint of a file — change a single byte and the fingerprint changes completely. The bundle records the fingerprint of the raw before/after coverage (1a17ee23..., 218d7936...), so anyone can re-hash those files and confirm the delta above was not edited after the run.
How to verify a bundle offline and replay it in the dashboard →
Step 4 — the weekly report
Countable deliverables, every number citing a bundle.
The weekly pilot report is deliberately boring: each row is a count, and each count names the exact artifact field it is read from — never an agent's summary of its own work.
| Report row | What it reports | Read straight from |
|---|---|---|
| Tests added this week | Count + the new test titles | report/coverage.json → tests[].title, set-diffed against last week |
| Coverage delta | Executable lines covered, this week vs last | report/coverage.json → summary |
| Flakiness | Flaky (passed on retry) vs failing | npx playwright test --reporter=json, stock Playwright |
| Behavioral-audit flags | New flags, each linking its filed issue | the audit pass — "none filed this week" when none |
| Next week's targets | The uncovered targets selected next | report/coverage.md — its !-marked lines |
Every artifact link in the report is one click from its evidence, so the buyer verifies the file, not our description of it.
Step 5 — honesty is the feature
We treat fabricated evidence as the one unacceptable outcome.
If a run produced no coverage, no kept tests, or no traces, the bundle's missing[] list records the path and the honest reason. Week one of a pilot has no prior week to diff against, so the report states the starting baseline — it does not invent a delta.
A test is kept only if it actually passes in the guest. In an earlier attempt of this very run, a candidate with a wrong selector failed and produced no kept test — the pipeline rejected it instead of keeping an inflated result.
The authoring step in this run used a stubbed reply, so its cost is real but meaningless ($0.00006) — and we say so, rather than dressing a plumbing test up as a claim about live-model quality.
Where this fits
Read next.
Why "tests passing" stopped being trustworthy, and what a verdict loop fixes.
Service-wrapped human QA versus an artifact you can re-run yourself.
Plain-English test authoring versus tests you keep in your own repo.