FINAL-Bench 五項開放科學挑戰由驗證器而非提交者決定評分
Five Open Science Challenges Where a Verifier, Not the Submitter, Decides the Score
FINAL-Bench 設有五項開放科學挑戰，參賽者提交分子、晶體、預測或解碼器，再由參賽者無法查看的驗證器評分；標準答案資料留在主辦方硬件上。各挑戰公開規則、權重及提交限制，但評分實作與答案保密，避免參賽者只針對公開評分器調整提交。LEADBOARD 為每個榜單提供已填妥的提示詞和可執行骨架；超導體挑戰明確允許 AI 編碼智能體參與，而所有分數均屬計算預測，並非實驗室測量。
TL;DR
Verifiers are one of the busiest topics in AI research this week: several papers on stronger verifiers appeared on the same day. The idea is simple. A model proposes, and a separate checker decides whether the proposal is right.
FINAL-Bench runs five open science challenges built on exactly that split. You submit a molecule, a crystal, a prediction or a decoder. A verifier you cannot see scores it, and the answer key never leaves the organiser's hardware.
| Challenge | What you submit | Verified against | Status |
|---|---|---|---|
| Open Discovery Challenge | A drug candidate (SMILES) | Target and counter-target structure prediction, ADMET, novelty | Seasons 2 to 4 open |
| Open Materials Challenge | A lithium electrolyte crystal | Oxidation and lithium-metal stability, novelty of use | Season 1 open to 2026-11-30 |
| Open Superconductor Challenge | A d-wave pairing estimate per 2D material | A many-body reference solver on top entries | Season 1 open to 2026-12-31 |
| Open Quantum Challenge | Hubbard energies or a surface-code decoder | A frozen private ground truth | Two seasons open into 2027 |
| LEADBOARD | Property predictions for held-out molecules | Labels that are never released | 21 boards open |
Why a hidden verifier matters
If the grader is public, an entry can be shaped against the grader instead of against the biology or the physics. Each challenge therefore publishes its rules, weights and entry limits, and keeps the scoring implementation and answer key private.
Built for agents as well as people
None of these needs special hardware for a first entry. LEADBOARD ships a filled-in prompt and a runnable skeleton for every board, and the superconductor challenge says plainly that AI coding agents can take part. An agent with a code tool such as Claude Code and a set of agent skills can read the rules, write the pipeline and submit.
Repositories
open-discovery-challenge · open-materials-challenge · leadboard · quantum-bench-harness
All scores in these challenges are computational predictions, not laboratory measurements. A high score is a hypothesis worth testing.
來源：DEV Community · dev.to