Measured comparison · August 2026
Greptile vs Ozone
Ozone caught 63 of 82; Greptile caught 35 — and both raised a similar number of false alarms, 23 against 22. Same noise, nearly twice the detection.
That is a cleaner result than most pairs in this test, because the usual trade-off is absent. Greptile is not buying precision with its misses; the false alarm counts are effectively level.
Greptile publishes its own benchmark, on which it reports 82%. Three independent re-runs by other vendors scored it between 24% and 45%, which is a good illustration of why we published our harness rather than only our numbers.
| Tool | Vulnerabilities caught | False alarms | |
|---|---|---|---|
| Greptile | 35 of 82 | 22 | |
| Ozone | 63 of 82 | 23 |
OpenSSF CVE Benchmark, 82 real JavaScript and TypeScript vulnerabilities, each also shown patched to count false alarms. Competitors' findings are their own published output (DeepSource, April 2026); the grading is ours, applied identically to all ten and blind to which tool produced a finding.
Both build a model of the repository
Greptile and Ozone share an architecture in broad terms: index the codebase, use that context when reviewing a diff, comment on the pull request. Neither is a file-at-a-time scanner, and both should in principle handle cross-file reachability.
The measured difference is in what each does with that context. On the same 82 vulnerabilities, Ozone caught 38 that Greptile missed, while Greptile caught 10 that Ozone missed — a net difference of 28.
On competing benchmark numbers
Greptile's published benchmark reports a 82% catch rate on a 50-pull-request set it assembled. Augment re-scored the same set and reported 45%; Macroscope reported 24%; Entelligence reported an F1 of 37.
We are not accusing anyone of bad faith — benchmark results move enormously with grader design, which we demonstrated on ourselves when a leaky grader prompt produced a fake perfect precision score for all nine tools we tested.
It is the reason every number on this page comes with a public harness, the raw findings, and every grader verdict. Re-run it and tell us if you get something different.
When to choose Greptile
- You want the pull request summaries and codebase question-answering it offers alongside review, which this benchmark does not measure.
- You are already deployed on it and the switching cost outweighs 28 additional catches on a JavaScript and TypeScript test set.
- Your codebase is in a language this benchmark does not cover, where our result gives you no information either way.
Common questions
- Is Greptile or Ozone better at finding vulnerabilities?
- On the OpenSSF CVE Benchmark, Ozone caught 63 of 82 and Greptile caught 35, with almost identical false alarm counts of 23 and 22.
- Why does Greptile's own benchmark show a higher score?
- Greptile reports 82% on a 50-pull-request set it built. Re-runs of that same set by Augment, Macroscope and Entelligence scored it between 24% and 45%. Benchmark results shift substantially with grader design, which is why our harness and raw data are published.
Check the numbers yourself.
The harness, the fixtures, every raw finding and every grader verdict are published. Re-run it and tell us if you get something different.