跳到正文
DEV Community· Savindu Thathsara·· 2 小時前AI 評分37

Hearthlen 智能體自行輸入網址

It Typed the Address Itself

AI 導讀

Hearthlen 的電腦操作智能體按指示在 Chromium 開啟 Debian 網站,整個流程用時 4613 ms;其 360M 模型無需預先知道 Debian 是甚麼。它利用螢幕閲讀器提供的控制項資訊選擇網址列,而非猜測頁面像素,並禁止輸出 shell 命令。

正文

Savindu Thathsara

It Typed the Address Itself

“Open the debian website” was the whole instruction. Four seconds later the ThinkPad was showing the Debian homepage, and nothing in the code knows what Debian is.

Savindu Thathsara, savithathsara.me. This is a Hearthlen note from the same laptop.

One action, then look again

The earlier path had a skill per job. Play a video, open an app, search the web. That works until the next sentence is something nobody wrote a function for. A computer-use agent does the other thing: look at the front window, choose one key or click, do it, and look again.

Hearthlen keeps a 360M model loaded only for that task. The model may open an installed program, press a short list of keys, type, or click a control that is actually on screen. It cannot emit a shell command. If the focused window is a terminal, the typing step is refused.

On this run Chromium was already in front. The steps were Ctrl+L, then https://www.debian.org, then Return. The window title was “Debian -- The Universal Operating System”. End to end: 4613 ms.

Why it did not click the page

Linux already tells a screen reader the name and place of each button. That list is a better eye than a screenshot on a laptop with no discrete GPU, and it is what the current open Linux computer-use tools read first. FeatherPad’s File, Edit, and Search menus showed up as real controls. Chromium showed none. For a browser, the honest move is the address bar, not a guessed pixel.

The grammar is doing a lot of the supervising. A 360M model will wander if you let it type a paragraph into the address bar. The next step is a stronger local model in the same loop, and a timed “hey Hearthlen” from a real voice, which is still the open check in the roadmap. The password-free browser window from the previous note is the window this loop types into.


Savindu Thathsara builds practical AI and automation in London. This series is the public log of an offline AI operating system he is writing and learning at the same time.

來源:DEV Community · dev.to