Simulation with realistic, interactive agents is a key task for autonomous driving. WOSAC scores whether your simulator matches real-world behavior distributions — not just one trajectory.
Waymo Open Sim Agents · 2025
Waymo's leaderboard is open but opaque locally. WOSAC Preflight runs the official 2025 metametric on one scenario — format gates, per-feature likelihoods, and traffic-light violation rates — before you burn GPU days or one of your 3 monthly test slots.
Simulation with realistic, interactive agents is a key task for autonomous driving. WOSAC scores whether your simulator matches real-world behavior distributions — not just one trajectory.
The test set has 44,926 segments. Official tutorials warn this "may take a significant amount of time." You get 3 test uploads per 30 days — and the leaderboard shows one meta-metric, not per-scenario diagnostics.
validate → score → doctor using official
waymo-open-dataset metrics (Docker on macOS). Receipt JSON +
this dashboard. Independent of any Waymax PR.
| Feature | Bucket | Likelihood | Weight |
|---|
| Method | Meta | Δ yours |
|---|