Measured comparison · August 2026
The best AI code review tools, measured on 82 real vulnerabilities
Two tools separate from the field. Cursor Bugbot caught 66 of 82 vulnerabilities and Ozone caught 63 — the next best, DeepSource, caught 48. Everything below that misses roughly half of what it is shown.
The tie-breaker is noise. Bugbot raised 42 false alarms on already-patched code; Ozone raised 23. If your team will investigate every flag, Bugbot's extra three catches are worth it. If they will start ignoring the bot after the tenth wrong answer, they are not.
The most widely deployed tool in this category, CodeRabbit, caught 16 of 82. That is the single most surprising number in this dataset, and the caveat below about how it was run matters.
| Tool | Vulnerabilities caught | False alarms | |
|---|---|---|---|
| Cursor Bugbot | 66 of 82 | 42 | |
| Ozone | 63 of 82 | 23 | |
| DeepSource | 48 of 82 | 6 | |
| Devin | 47 of 82 | 12 | |
| Codex | 46 of 82 | 16 | |
| Claude Code | 35 of 82 | 10 | |
| Greptile | 35 of 82 | 22 | |
| GitLab Duo | 30 of 85 | 15 | |
| CodeRabbit | 16 of 82 | 11 | |
| Semgrep CE | 12 of 82 | 6 |
OpenSSF CVE Benchmark, 82 real JavaScript and TypeScript vulnerabilities, each also shown patched to count false alarms. Competitors' findings are their own published output (DeepSource, April 2026); the grading is ours, applied identically to all ten and blind to which tool produced a finding.
How the test works
Each vulnerability is a real CVE with a real fix. We show a reviewer the file as it was when the flaw was live, as an ordinary pull request with a mundane title and no hint that anything is wrong. A good reviewer speaks up.
Then we show the same reviewer the patched version of that file. A good reviewer stays quiet. That second half is what most published benchmarks skip, and it is the half that decides whether a tool is usable day to day.
All figures come from the OpenSSF CVE Benchmark, maintained by the Linux Foundation: 82 real JavaScript and TypeScript vulnerabilities, each shown to every reviewer as an ordinary pull request, plus the patched version of each file to count false alarms. Competitors' findings are their own published output, released by DeepSource in April 2026. Only the grading is ours, and the grader is never told which tool produced a finding or whether the code it is reading is still vulnerable.
Why the two columns pull against each other
A tool can score perfectly on detection by flagging everything, and perfectly on precision by never speaking. Neither is useful. The question is what a tool does when it is unsure, and the ten tools here answer it very differently.
Detection-first, accepting noise: Cursor Bugbot (66 caught, 42 false alarms) and Greptile (35 caught, 22 false alarms).
Precision-first, accepting misses: DeepSource (48 caught, 6 false alarms) and Semgrep (12 caught, 6 false alarms). Both are quiet and trustworthy when they speak — but Semgrep misses 70 of 82.
Ozone sits at 63 caught with 23 false alarms: within three of the best detection in the test, at roughly half the noise.
What this test does not tell you
It covers JavaScript and TypeScript only, so it says nothing about Python, Go, Java, Rust or Solidity coverage for any tool here.
The CVEs are public and date from 2016 to 2021, so every model has likely seen them in training. That inflates all ten equally — the ordering holds, the absolute numbers are not real-world detection rates.
CodeRabbit and Claude Code were run through a CLI rather than as pull request reviews. Both are pull-request products. Their numbers here are very likely understated, and we would treat their results as a floor rather than a verdict.
It measures vulnerability detection, not review quality generally. A tool that scores badly here may still be excellent at style, structure or test coverage — which is what several of these products are actually built for.
When to choose another tool
- Choose Cursor Bugbot if your team is small enough to triage every flag and you want maximum detection. It caught three more than Ozone, and if you already live in Cursor the integration is one click.
- Choose DeepSource if false alarms are the thing that killed your last rollout. Six false alarms across 82 patched files is the cleanest result in the test, and it caught 48 — respectable, if well behind the top two.
- Choose CodeRabbit if breadth of review matters more than vulnerability detection. It is the most widely deployed tool in the category for reasons this benchmark does not measure, and the CLI caveat above applies to its score.
- Choose Semgrep if you want deterministic, rule-based scanning you can audit and version — a genuinely different tool from everything else on this list, and free.
Common questions
- Which AI code review tool catches the most vulnerabilities?
- Cursor Bugbot caught 66 of 82 real vulnerabilities, followed by Ozone at 63 and DeepSource at 48. Bugbot also raised the most false alarms, 42, against Ozone's 23.
- Is this benchmark independent?
- The dataset is the Linux Foundation's OpenSSF CVE Benchmark, and the competitor results are DeepSource's own published output. Cecuro ran Ozone and re-graded all ten tools with one grader that is blind to which tool produced a finding. The harness and every raw finding are published so the run can be reproduced.
- Why did CodeRabbit score so low?
- CodeRabbit caught 16 of 82 in this run. DeepSource's methodology states it was run through a CLI against the filesystem rather than as a pull request review, which is not its native mode, so the figure is likely understated.
Check the numbers yourself.
The harness, the fixtures, every raw finding and every grader verdict are published. Re-run it and tell us if you get something different.