跳到正文
Hacker News:AI 熱帖·· 4 小時前AI 評分37

DishwasherBench:前沿模型能否把碗碟妥善放入洗碗碟機?

DishwasherBench – Can Frontier Models Stack a Dishwasher?

AI 導讀

DishwasherBench 讓人類和 AI 在按作者自家 Bosch 洗碗碟機校準的模擬環境中擺放碗碟,測試前沿模型能否完成這項任務。項目仍在開發中,模擬環境尚待完善,尚未進行完整基準測試。

正文
DishwasherBench – Can Frontier Models Stack a Dishwasher? (github.com/composo-ai)
30 points by mlukewizard 1 hour ago | hide | past | favorite | 9 comments

The obvious fork is "can frontier models stack a dishwasher to my wife's exacting standards".

I can't. So that would surely be proof of AGI.


"In every partnership, there is a person who stacks the dishwasher like a Scandinavian architect and a person who stacks the dishwasher like a raccoon on meth." - Someone online.

At home I'm the architect, my wife's the raccoon. I don't actually believe that you can stack the dishwasher wrong as such, but you can make it hard for yourself to unload, if you don't sort the items correctly.


Never have truer words been spoken, I too score poorly on DishwasherBench wife mode. Drives her up the wall.


At our place it's opposite, I guess when you do it often enough you get a preferred method though a relentless optimization loop, which in the end ensures it's all clean, the plates are well separated only in 1 particular order, the glasses are placed to the left so all small pans fit at the right, deep plates are placed half over cutlery rack so they are supported and more fit in, small plates in front so all lids fit before them... etc etc etc.


Inspired by Joma Tech's new YouTube video

Stacked loads are run against a dishwasher simulation calibrated to my own Bosch dishwasher

Playable by both humans and AI

Currently a WIP, simulations need refining before running a full benchmark, but keen to hear some feedback!


This is so cool!

Imagine creating real simulated env for every physical task and using RL to train robots. That will be a game changer.


sure the code isn't hand-written and it's a work in progress, but it's designed to be modelled and calibrated around my own dishwasher so it's verifiably non-slop

來源:Hacker News:AI 熱帖 · news.ycombinator.com