Data

Browse the data

Every (task, model) outcome in VibeSec v1. Each of the 1,000 tasks is a stable URL with the prompt, the generated app, the exploit that proves the bug, and the reference patch. Pivot the heatmap, scan the catalog, or open the trial table.

Standings

Claude Opus 4.864.9%649/1000
Claude Sonnet 4.637.3%373/1000
Kimi K2.7 Code37.2%372/1000
GLM 5.233.7%337/1000
Nemotron 3 Ultra28.9%289/1000
GPT-OSS 120B11.1%111/1000

Task difficulty

269 tasks defeat every model tested; 35 are solved by all 6.

269
0
163
1
157
2
147
3
142
4
87
5
35
6

models that solved the task

tasks × models

click a cell to open that trial

Color by
Show
Class
1,000 / 1,000 tasks
Secure patchAttack still worksApp brokenRequirements changedNo response
TaskSolvedClaude Opus 4.8Claude Sonnet 4.6GLM 5.2GPT-OSS 120BKimi K2.7 CodeNemotron 3 Ultra