智能合約掃描工具在 20 個已知漏洞中找到 14 個：自行重算並瞭解其不足

We sell a deep scan for Solidity repositories. On 2 November 2026 we measured it for the first time on five public audits from the EVMbench set: it found 14 of the 20 known High findings. On the same five audits the leader's own report finds 15 under our judge copy (16 under the benchmark's own verdict on one finding), and the older, simpler route of our shop found 8. EVMbench is the open benchmark of known High findings from past audit contests, and the leader is Azimuth, the best published run on it that we know of.

The folder with the verdicts, the exact reports the judge read, a table and a script that recounts everything is public: results/2026-11-02-shop-setup-five-audits in github.com/holistis/bug-bounty-intelligence-mcp. Clone it and run node check.mjs. No network and no key. It recomputes every count from the 20 verdict files and checks the hashes. It checks the counting, not whether the judge's verdicts are right.

How it was measured

One run per audit, from a script on our own machine and not through the shop, with the scan code, model and thinking limit the shop uses, and a flat cost limit of 1.50 dollars per audit. The cheaper 1 and 5 USDC routes of the shop are older and are not measured here. The judge is a copy of the EVMbench detect grader (gpt-5, high reasoning effort, the benchmark's grading prompt, one call per known finding), run through OpenRouter and not through the benchmark's own harness. When we checked that copy on the leader's report it gave the same verdict as the benchmark's own answer on 30 of 31 findings. The five scans cost 5.88 dollars in total.

On phi our run found H-06. Our judge copy says the leader's report misses it, and the benchmark's own grader counts it as found there. It is the one finding where the copy and their grader disagree. On size we found one of four.

What this does not tell you

• Five audits, one run each. One finding moves the share by five points, and the judge does not answer the same every time.
• They are development audits from 2023 to 2025, so they may be in the training data of the model behind the scan. This is not a result on new contracts.
• The hardest audit in the set, noya, is not included.
• Two runs, size and nextgen, hit our 1.50 dollar limit and were stopped. We have not measured what the shop's own limits would do on them.
• The runs did not read every file: 14 of 20 on phi, 48 of 76 on nextgen, 37 of 39 on size. A bug in a file that was not read cannot be found.
• Four of the six findings we gained over the older route come from one audit, forte, where the older run had been stopped by its cost cap and counted as zero.

A mistake we made on the way

Our first public number for the shop scan, 8 of 20, was labelled as the scan we sell. It was an earlier route. We corrected the text and kept the old text visible, in results/2026-11-02-our-scan-under-the-benchmark-judge/README.md.

If a number does not match

If a number in the folder does not match when you run the check, say so in issue 41 of the repository and we will correct it and keep the old text visible. Our scoreboard at tokenizen.nl/scoreboard lists every measurement with the judge named next to it.

Best regards,
Abdellah
github.com/holistis