Security auditor OpenZepplin found that EVMbench’s dataset contains training data leaks and at least four invalid high-severity vulnerability classifications.
Blockchain security firm OpenZeppelin says it has found methodological flaws and data contamination in its audit of OpenAI’s new artificial intelligence benchmark for blockchain security, EVMbench.
EVMbench was launched in partnership with crypto investment firm Paradigm in mid-February. It was built to evaluate how well different artificial intelligence models can identify, patch, and exploit smart contract vulnerabilities.
In an X post on Monday, OpenZeppelin said it welcomed the initiative but recently decided to put EVMbench “through the same scrutiny” it applies to all the protocols it helps secure, including the likes of decentralized finance heavyweights Aave, Lido and Uniswap.
