Data
Browse the data
Every (task, model) outcome in VibeSec v1. Each of the 1,000 tasks is a stable URL with the prompt, the generated app, the exploit that proves the bug, and the reference patch. Pivot the heatmap, scan the catalog, or open the trial table.
Standings
Claude Opus 4.864.9%649/1000
Claude Sonnet 4.637.3%373/1000
Kimi K2.7 Code37.2%372/1000
GLM 5.233.7%337/1000
Nemotron 3 Ultra28.9%289/1000
GPT-OSS 120B11.1%111/1000
Task difficulty
269 tasks defeat every model tested; 35 are solved by all 6.
2690
1631
1572
1473
1424
875
356
models that solved the task
tasks × models
click a cell to open that trial
Color by
Show
Class
1,000 / 1,000 tasksSecure patchAttack still worksApp brokenRequirements changedNo response
TaskSolvedClaude Opus 4.8Claude Sonnet 4.6GLM 5.2GPT-OSS 120BKimi K2.7 CodeNemotron 3 Ultra