ViBench · native product benchmark

ViBench Native MVP-96

A planned 24 × 4 matrix. The 24-app Codex construction treatment is complete; the other 72 cells remain missing. Missing results never count as zero.

Summary

Build and evaluation have different credential rules. Read the process method.

Coverage is explicit: missing and unevaluated cells display an em dash. They are not scores.

Treatments and runner readiness

Treatment scores

The primary value uses all 74 pinned plans. Available-case values are provisional and are not leaderboard scores.

24 × 4 result matrix

Per-app results

All 74 plan results

The wedding/test1 anomaly remains in the primary universe. The separate 73-plan value is an audit diagnostic only.

Per-cell records and construction files

Only exact file hashes in the checked-in reviewed-body allowlist can be fetched as inert JSON strings, and heuristic guards can still veto them. New or changed bodies are withheld by default. This is not a universal secret or PII detector. Candidate HTML and JavaScript are never loaded as documents or executed. Screenshots are not generated; raw private evidence is withheld.

Prompts and upstream inputs

Every cell receives the same canonical goal prompt plus its app's pinned MVP PRD and public assets. Test plans are evaluation inputs, not construction inputs.

Prompt SHA-256:

Methodology

Process

Each build agent uses its native subscription login and native tools. The copied ViBench evaluator scores the result. It does not build the app. We use the same fixed evaluator for every full score. The evaluator uses its own API key. We do not give this key to a build agent. We have not yet used the evaluator to score a full ViBench test plan. In the first private test, the evaluator and the candidate app can run in the same container. This setup does not prove that the evaluator is isolated. We publish a comparable score only after the result passes all publication checks.

Scores

Upstream points anomaly

Missing cells

Public artifacts

Data

Download the normalized public data

data.json SHA-256: c33f1e08c836517d6fa697f32966db00c4175fd5c1dad904e4370f7eefbccf60

artifact index SHA-256: 77b322706dd951114974fe17f6b9cfb64570b1b3a9220775e753a065a94965be

Candidate output bodies publish only when their exact cell, path, and SHA-256 have been reviewed and allowlisted and the heuristic vetoes pass. Every file still retains exact hash-bound tree metadata.