跳到正文
The Elec· Hyun-Seon, Park·· 13 天前AI 評分51

Cogent Security 的 VR-1.1 在 LiveCVEBench 準確率高於 Claude Code 和 Codex

Security AI Agent Outperforms GPT and Claude

AI 導讀

Cogent Security 的 VR-1.1 在 LiveCVEBench 的 190 個 CVE 測試中取得 57.9% 準確率,超過 Claude Code(Claude Opus 5.5)和 Codex(GPT-5.6 Cyber)的 45.3%。

正文

LiveCVEBench leaderboard as of Nov. 6, 2026. (Source: LiveCVEBench)

LiveCVEBench leaderboard as of Nov. 6, 2026. (Source: LiveCVEBench)

LiveCVEBench leaderboard as of Nov. 6, 2026. (Source: LiveCVEBench)
LiveCVEBench leaderboard as of Nov. 6, 2026. (Source: LiveCVEBench)

A security-focused artificial intelligence (AI) model developed by a startup is drawing attention after outperforming AI models from Anthropic and OpenAI.

VR-1.1, developed by California-based AI security startup Cogent Security, achieved 57.9% accuracy across 190 Common Vulnerabilities and Exposures (CVEs) on LiveCVEBench. That exceeded the 45.3% recorded by Claude Code powered by Anthropic's Claude Opus 5.5 and the 45.3% achieved by Codex based on OpenAI's GPT-5.6 Cyber by more than 12 percentage points.

CVE is a system of unique identifiers assigned to publicly disclosed software and hardware security vulnerabilities. LiveCVEBench is a public benchmark that evaluates AI coding agents' ability to independently analyze and remediate CVEs.

However, Cogent VR-1.1 lagged in cost efficiency. Successful tasks required an average of approximately 2.23 million tokens and 31 turns, compared with about 380,000 tokens and 8.4 turns for Claude Code. Although VR-1.1 delivered higher accuracy, it was less cost-efficient. In responding to security threats, however, performance takes priority over cost.

When accuracy scores are tied, the agent using fewer tokens ranks higher. Codex, which matched Claude Code's accuracy, consumed approximately 684,000 tokens and 15.3 turns, giving Claude Code the advantage. A turn represents one interaction in which a model responds to an input. In agent workflows, it refers to a cycle in which the model performs an action, such as opening a file, executing a command or modifying code, and receives the result. Fewer turns indicate faster problem resolution.

According to Cogent, VR-1.1 begins by analyzing vulnerability reports and source code, then generating proof-of-concept (PoC) test code to reproduce the defect. Once the vulnerability is reproduced, it traces the affected code paths, applies a targeted patch and conducts a separate review to verify that no new security weaknesses have been introduced. It then automatically generates regression tests to prevent the same bug from recurring after future code changes.

(Source: LiveCVEBench)
(Source: LiveCVEBench)

Cogent also claims that every patch generated by VR-1.1 includes diagnostic results, PoC code and records of executed tests, allowing security teams to review and approve fixes quickly. Cogent Security was co-founded by two former employees of Abnormal Security, an AI-based email security company, and a third founder with experience at Coinbase and Blackstone.

However, the results represent only one security benchmark and do not establish overall superiority. The nature of LiveCVEBench also makes it difficult to evaluate the latest AI models, meaning results could differ when newer models from OpenAI and other developers are tested against newly disclosed CVEs.

This limitation stems from one of LiveCVEBench's central principles: preventing AI models from solving benchmark tasks by memorizing previously published answers. Newer models are more likely to have incorporated recent commits from open-source repositories and CVE patch records into their training data. To ensure fair comparisons, LiveCVEBench uses baseline model versions trained before the relevant CVEs were disclosed.

來源:The Elec · thelec.net