跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 6 天前AI 評分48

Accelerate Bench:評估智能體進行應用程式層級程式最佳化的表現

Accelerate Bench: Evaluating Agents for Application-Level Program Optimization

AI 導讀

Accelerate Bench 是一個評估編碼智能體進行應用程式層級程式最佳化的基準測試,涵蓋 8 個應用領域的 32 項科學工作負載。研究評估了 16 個編碼智能體,排名前列的兩個智能體在至少 22 項任務中取得 2× 或以上加速,同時須維持程式正確性。31 項任務可透過 Harbor Hub 使用,完整版本則在 Hugging Face 提供。

正文

Hi Hugging Face community! We’re introducing Accelerate Bench, a benchmark for evaluating coding agents on application-level program optimization.

🧪 32 scientific workloads across 8 application areas
Agents optimize complete CPU applications, including GROMACS, OpenFOAM, and Quantum ESPRESSO. Each task provides a working program, a fixed workload, correctness requirements, and a compute budget. Agents have eight hours to decide where and how to optimize.

✅ Performance gains must preserve correctness
Public tests support iterative development. A separate verifier rebuilds the final submission, checks hidden inputs, and measures its runtime against the original program.

📊 16 coding agents evaluated
The two leading agents achieve at least 2× speedups on 22 or more of the 32 tasks. Controlled experiments show why application-level coordination matters: the same source code can perform very differently under different build and runtime settings, and the benefit of one optimization can depend strongly on another.

🛠️ Try it with your own agent
31 tasks are available directly through Harbor Hub, with all 32 available through the full release on Hugging Face.

We welcome evaluations, reproductions, and discussion from the community!

🌐 Project: https://acceleratebench.com/
🤗 Dataset: https://huggingface.co/datasets/AdAstraAbyssoque/accelerate-bench
🚀 Harbor Hub: https://hub.harborframework.com/datasets/accelerate-bench/accelerate-bench

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co