Benchmark: $97M in DeFi Exploits Preventable by Specialized AI **TL;DR** - **92% detection rate**: Cecuro Security Agent detected vulnerabilities in 83 of 90 real-world exploited contracts - **$96.8M protected**: Covered exploitable value across the benchmark dataset - **13x gap**: Purpose-built AI protected 13x more value than general-purpose AI ($7.5M) - **Same model**: Both agents use GPT-5.1. Difference is in application layer and domain expertise. - **Major catches**: GMX ($42M), Cork Protocol ($12M), Resupply ($9.6M), yETH ($9M), 1inch Fusion ($5M), FPC ($4.7M) - **Open sourced**: Full benchmark dataset and baseline agent code released on [GitHub](https://github.com/Cecuro/defi-vuln-benchmark) - **Key finding**: AI exploit capability doubles every 1.3 months. Defense must match offense. [Start your AI-powered audit →](https://app.cecuro.ai) --- ## Abstract AI exploit capability is doubling every 1.3 months (1), creating a rapidly escalating threat for smart contract projects lacking adequate security review. To quantify both the threat and potential for AI-driven defense, we benchmark Cecuro, an agentic smart contract audit system, against a standard frontier coding agent (GPT-5.1 based) on 90 real-world exploited smart contracts representing $228M in losses, all post-September 2024 to control for training data contamination. Our purpose-built system detected vulnerabilities in 92% of contracts (83/90), covering $96.8M in exploitable value, compared to a 34% detection rate and $7.5M coverage for the standard agent, a 13x gap in protected value. These results demonstrate that deep domain expertise and agent optimization can 2-3x vulnerability detection performance over baseline agentic code review on the same underlying model. Separately, frontier agents now execute end-to-end exploits on 72% of known vulnerable contracts (2), underscoring that offensive capability is advancing in parallel. The benchmark dataset and the baseline agent are open-sourced; the full Cecuro Security Agent is not, given the risks of making autonomous exploit tooling publicly available. --- ## Introduction In 2025 crypto theft amounted to $3.4 billion, of which $1.5 billion came from a single compromise of Bybit in February (5). These numbers point to a massive issue in the security landscape of smart contracts that is not keeping up with the requirements of the financial systems it powers. Protecting smart contracts and Web3 cybersecurity is very challenging with high scarcity of expert knowledge. The vulnerabilities that cause the largest losses are rarely obvious. They could be a well hidden reentrancy path that is only reachable under specific liquidity conditions, an accounting flaw that surfaces only when two protocols interact in an unexpected way, an access control gap that opens only after a governance vote. Identifying these vulnerabilities requires deep expertise in programming languages, system governance mechanisms, and DeFi protocol economics, a skillset that is scarce and highly sought after. While professional human audits offer some help, they are expensive (ranging from $15,000 to over $300,000), take several weeks, and only cover the codebase at a single point in time. Consequently, many projects lack full audit coverage, and some skip the process entirely. In December 2025, Anthropic's red team published research demonstrating that AI agents could steal $4.6 million from live smart contracts in a controlled environment and that this capability had been doubling roughly every 1.3 months (1). Shortly after, researchers from OpenAI, Paradigm, and OtterSec released EVMBench (2), showing frontier agents could execute end-to-end exploits on 72% of known vulnerable contracts extracted from code4rena audit competitions before mitigations were applied. EVMBench contains three different challenges: Discover (find the vulnerability), Exploit (write a working proof-of-concept to exploit it), and Patch (fix it), testing different aspects of model capabilities in securing smart contracts. EVMBench uses a simple system prompt across the leading CLI tools such as Codex (9) and Claude Code (8). This study instead measures the difference between generalized AI coding assistants and the capabilities of purpose-built security agents designed for the designated tasks on 90 real-world contracts that were actually exploited between October 2024 and early 2026, representing $228 million in verified losses. This dataset builds on top of the Anthropic SCONE-bench dataset and adds more recent exploits to increase the sample size. What we want to show is not if the agents could have exploited the contract after the fact, but whether it could have flagged the vulnerabilities before attackers did. --- ## The Benchmark We have done this by evaluating two agentic systems, **Cecuro Security Agent** and **Basic Agent**, head to head on a new benchmark consisting of 467 real-world smart contract exploits drawn from two sources. The first set (V1) is sourced from (1), the dataset built by Anthropic's red team for their December 2025 smart contract security research. This is a curated collection of high-impact EVM exploits originating from DeFiHackLabs (3) exploit database. The second set (V2) of the dataset was processed using mechanisms similar to Anthropic's methods to ensure the root cause is identifiable from the exploited contract address. All contracts are EVM-based, deployed on BSC, Ethereum, Optimism, and Polygon, and were evaluated using Foundry fork simulation environments, enabling deterministic replay of on-chain state. The dataset used for all evaluations is restricted to the 90 exploits occurring after September 30, 2024, controlling for the GPT-5.1 training cutoff and ensuring results reflect genuine detection capability rather than memorized exploit patterns. Dollar values represent actual reported losses from the original attacks. The dataset spans the full range of vulnerability complexity: simple access control failures detectable with pattern matching; multi-step accounting flaws requiring hypothesis testing across protocol logic; and cross-contract systemic vulnerabilities that require reconstructing protocol economics from first principles. Total reference losses: **$227.9 million**. The full benchmark dataset and evaluation framework are open-sourced. The **Cecuro Security Agent** is not, given the risks of making autonomous security tooling publicly available to malicious actors. The basic agentic baseline is open-sourced as a reproducible reference point. ### Systems Evaluated **Cecuro Security Agent**: A multi-phase agentic pipeline built specifically for smart contract security. The system applies exhaustive Web3-specific domain knowledge, such as understanding of lending protocol mechanics, AMM price manipulation vectors, flash loan interaction patterns, fee-on-transfer token edge cases, and cross-contract callback risks that general-purpose agents lack. **Basic Agent**: A standard ReAct-style agentic loop using GPT-5.1 with access to source code and basic file tools, but without domain-specific security methodology, verification passes, or DeFi protocol knowledge. This approximates what a development team would obtain by connecting a frontier model to their codebase with minimal configuration, the "just use ChatGPT" baseline. Both systems operated within identical environments using a local RPC node fork (Foundry). The fork was pinned to the specific block height immediately preceding the exploit, providing a sandbox for vulnerability validation. For each challenge both systems were given access to the source code extracted from Etherscan (11) and tasked with identifying security vulnerabilities. Results were scored against the known reference exploit: **FOUND** (vulnerability correctly identified) or **NOT FOUND**. --- ## Evaluating the Agents In Figure 1, we see a noticeable performance difference between the **Basic Agent** and the **Cecuro Security Agent**. Cecuro covers vulnerabilities leading to **$96.8M** of the $228M dataset, which is more than a **13x gap** in protectable value compared to the Basic Agent, which covers **$7.5M**.  _Figure 1: Detection rate and value protected across 90 real-world exploited smart contracts ($228M total reference losses, 2024-2026). Both agents use the same underlying frontier LLM (GPT-5.1 with high reasoning). The difference in performance is driven entirely by application layer and agentic harness._ The gap in performance reflects the difference in vulnerability complexity and the monetary value assigned to them. The Basic Agent identifies straightforward and obvious vulnerabilities in simpler contracts. The high-value, high-complexity exploits, the ones that really drive DeFi's loss statistics, require structured, multi-phase investigation that general-purpose agents do not naturally consider. Looking beyond the absolute dollar amount, there is also a noticeable difference in detection rate of 2-3x (92% vs. 34%). The financial impact of the exploits follows a highly skewed distribution. As evidenced by the data below, a small number of findings are responsible for the overwhelming majority of the total value: just 6 findings identified by the Cecuro Security Agent account for a substantial $82.3 million. ### Major Catches - **GMX: $42M loss.** Access control flaw in position management across a multi-contract system - **Cork Protocol: $12M loss.** Logic flaw in yield accounting - **Resupply: $9.6M loss.** Multi-protocol lending interaction vulnerability - **yETH: $9M loss.** Unsafe math in withdrawal logic - **1inch Fusion: $5M loss.** Arbitrary calldata injection in Yul-optimized settlement - **FPC: $4.7M loss.** Burn-on-transfer accounting flaw These are not simple or mechanical bugs. They required reasoning across multiple contracts, understanding DeFi protocol economics, and recognizing that patterns appearing safe in isolation can become exploitable under specific market conditions. Several of the contracts in the dataset had previously received professional security reviews without the vulnerability being identified. --- ## Detection is Harder than Exploit Exploitation and detection look like two sides of the same problem, but they are structurally very different tasks for an agent. In exploit mode, the agent can get immediate feedback from the environment if the exploit works by inspecting the code on chain. The task is hard, but success is unambiguous, which makes developing successful agents for benchmarks like SCONE-bench less challenging. Detection has no equivalent signal. The agent reads code, forms hypotheses, and flags potential issues with no reliable "simple" way to confirm in real time whether a finding is real. Across our evaluations, three challenges consistently limited agent performance: - **No verifiable feedback:** PoC generation is one partial answer, and recent work like PoCo (6) and SmartPoC (7) shows real progress. But these systems start from a known vulnerability. The harder problem is discovery itself, knowing where to look before you know what you are looking for. - **No systematic coverage:** Without domain-specific guidance, agents tend to follow shallow paths and spend their budget on surface-level patterns. We saw this directly in our testing. In some runs the basic agent traced a peripheral contract for most of its budget and never reached the vulnerable function. Starting conditions and initial direction explained more of the variance in results than model capability did. - **Context saturation and output variance:** A pattern we observed across runs is that agents tend to treat the review as complete once a handful of findings are flagged, even when large parts of the codebase remain unexamined. This persists even with explicit planning mechanisms like todo tools and system reminders in place. It is why pass@k metrics matter so much for this task. In our benchmark each agent ran twice per contract, and internal testing shows meaningful gains from additional passes. Anthropic uses pass@8 in their SWE-bench evaluations for the same reason. EVMBench confirms the variance directly: detection performance for GPT-5.3-Codex (high reasoning) ranges from 26.7% to 41.8% across runs. EVMBench's hint experiment ties this together well. Providing agents with file-level guidance on where to look lifts Patch task performance from roughly 40% to 90%. The capabilities are largely there. The agents are just not being directed to use them. The bottleneck is not model intelligence. It is the conditions under which the model operates. This dataset covers only a small subset of real-world exploits, drawn from cases that have been documented with PoCs by DeFiHackLabs. There are many attack vectors and risks in the DeFi space not captured here, and AI is not a solution to all of its security challenges. The shortage of experienced security researchers and auditors remains a real problem. But based on the results shown here, we believe continuous AI review should be standard practice. Several of the exploited contracts discussed in this report had previously passed professional human audits, yet the vulnerability went unnoticed. We believe AI offers a solution, not by replacing expert review, but by optimizing its focus. By automatically surfacing an estimated 92% of detectable vulnerabilities, AI frees human experts to concentrate their efforts on the cases that truly require their in-depth attention. --- ## The Economics The cost structure of smart contract exploitation has fundamentally changed. Anthropic's research puts the average cost of an AI-powered exploit attempt at roughly $1.22 per contract. At that price, a motivated attacker can scan thousands of contracts for under $2,000, and the capability behind each scan doubles every 1.3 months. Two years ago, exploiting a smart contract required deep technical expertise, custom tooling, and significant time investment. That barrier is gone. Anyone with API access and basic domain knowledge can run the same class of analysis that previously required a skilled security researcher. The floor has dropped from months of human effort to minutes of computation. This inverts the security calculus for every deployed contract. A traditional audit covers the codebase as it existed at one moment, at a cost that prices out most smaller protocols entirely. Meanwhile, the attacker's marginal cost per target approaches zero, and their tooling improves on a timeline measured in weeks, not years. The asymmetry is stark: attackers benefit from AI automatically, because exploitation is self-verifying. Funds moved or they didn't. Defense requires deliberate investment in purpose-built systems, because detection demands structured investigation, false positive filtering, and domain-specific reasoning that general-purpose models don't perform on their own. Projects that haven't adopted AI-powered security review aren't competing against last year's threat. They're exposed to an offense capability that compounds monthly, at a cost the attacker barely notices. The question facing every Web3 project is no longer whether AI changes their security posture, it already has. The question is whether they've built the defense side to match. --- ## Conclusion Recent research from Anthropic and OpenAI showed that AI can find and exploit smart contract vulnerabilities at low cost, and that capability is improving with every model release. This report adds to that picture. The **Cecuro Security Agent** flagged 92% of real-world exploited contracts in our dataset, covering $96.8M of the $228M in reference losses. A general-purpose agent on the same contracts covered $7.5M. Both run on the same underlying model. The gap is entirely in what is built around it. The offense economics are already compelling. At $1.22 per contract, the cost Anthropic showed in their SCONE-bench paper (1), the cost of running a systematic campaign across DeFi is essentially nothing. What changes the calculus further is specialization. If a purpose-built defensive system, like Cecuro Security Agent, can surface $96.8M in protectable value from this dataset, a purpose-built offensive system pointed at live contracts with real liquidity is a serious threat. An attacker who knows the expected return can justify spending far more than $1.22 per contract, and DeFi has no shortage of targets. This is why detection matters. **Projects not using AI for defense are exposed in a way that simply was not true a year ago**. AI-powered security review is available today at a fraction of the cost of a single audit, covering the vast majority of real-world exploit classes. Every DeFi project sits on one side of the 92% or the other. --- ## Appendix **Benchmark**: 90 EVM contracts exploited between October 2024 and early 2026. V1 contracts sourced from SCONE-bench; V2 contracts from Cecuro's filtered subset of DeFiHackLabs. Deployed on BSC, Ethereum, Optimism, and Polygon. Evaluated using Foundry fork simulation. **Date filter**: All contracts post-2024-09-30, controlling for the GPT-5.1 training cutoff. **Dollar values**: Actual reported losses from original real-world exploits. Not simulated extraction values. **Evaluation**: Binary detection task scored against known reference exploit mechanism. **Open source**: The benchmark dataset, evaluation framework, and basic agentic baseline are open-sourced on [GitHub](https://github.com/Cecuro/defi-vuln-benchmark). The Cecuro Security Agent is not released publicly due to the risks of making autonomous security tooling available to malicious actors. **Limitations**: - Detection quality evaluated against reference exploit findings, which may not capture all vulnerabilities in a given contract - Retrospective design: systems analyzed contracts where the exploit is known, which may advantage pattern-recognition over genuinely novel discovery - False positive rates not measured in this benchmark; precision is a separate and critical evaluation dimension - Single-pass evaluation; multi-pass detection rates would likely be higher - Cecuro results reflect a purpose-built system; basic baseline reflects minimal configuration of a general-purpose frontier model - One dimension this report does not measure is precision: what fraction of flagged findings are genuine versus false positives. Recall is one axis of utility, signal quality is the other, and it is an equally important topic we are actively pursuing at Cecuro, but there are no high quality public datasets available for this --- ## References (1) Anthropic Red Team. "Smart Contract Security Research." December 2025. [Online]. Available at: [https://red.anthropic.com/2025/smart-contracts/](https://red.anthropic.com/2025/smart-contracts/) (2) OpenAI, Paradigm, and OtterSec. "EVMBench: Evaluating Frontier AI on EVM Smart Contract Exploitation." 2026. [Online]. Available at: [https://cdn.openai.com/evmbench/evmbench.pdf](https://cdn.openai.com/evmbench/evmbench.pdf) (3) SunWeb3Sec. "DeFiHackLabs: DeFi Hacking Incidents Archive." GitHub. [Online]. Available at: [https://github.com/SunWeb3Sec/DeFiHackLabs](https://github.com/SunWeb3Sec/DeFiHackLabs) (4) OpenAI. "GPT-5.1 Model." [Online]. Available at: [https://developers.openai.com/api/docs/models/gpt-5.1](https://developers.openai.com/api/docs/models/gpt-5.1) (5) Chainalysis. "Crypto Hacking Hits $3.4B in 2026." [Blog Post]. Available at: [https://www.chainalysis.com/blog/crypto-hacking-stolen-funds-2026/](https://www.chainalysis.com/blog/crypto-hacking-stolen-funds-2026/) (6) V. Andersson, S. Bobadilla, H. Hobbelhagen, and M. Monperrus, "PoCo: Agentic Proof-of-Concept Exploit Generation for Smart Contracts," _arXiv_, Nov. 2025. Available at: [https://arxiv.org/abs/2511.02780](https://arxiv.org/abs/2511.02780) (7) L. Chen, R. Yan, T. Wong, Y. Chen, and C. Zhang, "SmartPoC: Generating Executable and Validated PoCs for Smart Contract Bug Reports," _arXiv_, Nov. 2025. Available at: [https://arxiv.org/abs/2511.12993](https://arxiv.org/abs/2511.12993) (8) Anthropic. "Claude Code." Accessed Feb. 19, 2026. Available at: [https://claude.com/product/claude-code](https://claude.com/product/claude-code) (9) OpenAI. "Codex." _GitHub_. Accessed Feb. 19, 2026. Available at: [https://github.com/openai/codex](https://github.com/openai/codex) (10) Cecuro. "Agentic Smart Contract Auditing." Accessed Feb. 19, 2026. Available at: [https://cecuro.ai/](https://cecuro.ai/) (11) Etherscan. "Etherscan". Accessed Feb. 19, 2026. Available at: [https://etherscan.io/](https://etherscan.io/) --- ## About Cecuro Cecuro provides AI-powered smart contract audits in hours, not weeks. Our multi-agent security system analyzes code using domain expertise and structured methodology, achieving 92% detection rate on real-world vulnerabilities. - ⚡ **Speed**: Audits take ~8 hours - 💰 **Cost**: 90% cheaper than traditional audits - 🔍 **Coverage**: All chains, all smart contract languages - 🤖 **Proven**: 92% detection rate on $228M benchmark [Start your audit now →](https://app.cecuro.ai) | [View benchmark on GitHub →](https://github.com/Cecuro/defi-vuln-benchmark)