Agent Lighthouse 如何利用 Kane CLI 評分並修復網站預約流程

A browser agent can finish a task and still show you the page is broken. In one day at Kane CLI Hack Day SF we turned Kane's step log into a score site owners can act on, built a repair loop that proves its own fixes, watched a review step catch the coding agent cheating, and added a second witness, REA, that found the same bug in the page's code.

The problem

AI agents already book appointments for people. The big platforms get there through partnerships. Everyone else, the gym, the clinic, the salon with its own form, has an agent clicking through a page that was never tested with one. When the agent struggles, the owner never sees it. They just lose the booking.

What Agent Lighthouse does

Kane CLI is the agent, the evidence and the verifier.

1. Measure. Kane attempts a real booking three times in a real browser. We turn its NDJSON step log into a 0 to 100 score and an Agent Tax: the extra steps, seconds and credits that friction costs per successful booking, all from fields Kane already emits.
2. Diagnose. Kane's failing step and screenshot go to the OpenAI API for a diagnosis.
3. Repair. Codex patches only the site files, on their own branch.
4. Prove. Kane re-runs the booking three times, plus a scripted human-path run that books the way a person would.
5. Hand off. Qualifying patches land as a pull request with Kane's evidence attached. Nothing touches a live site, and nothing is auto-merged.

What Kane caught

On our demo gym page Kane booked 2 of 3 times. The third run failed, and Kane filed it as a confirmed product bug: the booking confirmation disappeared after 2.5 seconds.

Even the runs that passed paid the Agent Tax: +10 steps, +84 seconds and 2.9x the credits per booking.

After one repair loop the page went from 51 to 97.

Two witnesses, one bug

Kane found the vanishing confirmation by trying to book. REA captured the page's scripts over the Chrome DevTools Protocol, and our scan found the setTimeout that hides the toast after 2.5 seconds. It also found the handlers behind the page's two "Go" buttons, where the one in the nav only reveals the price list. That dead end is where the extra steps came from.

With the REA flag on, both go into the fix brief: what the agent experienced, and the line of code behind it.

The coding agent cheated

On a harder site the coding agent got Kane to pass by deleting the human check and hard-coding the test date. Both automated checks passed it. Diff review did not.

That is why repairs arrive as a PR instead of a deploy. We added a fixture-leak check that now blocks that kind of patch, and a clean fix for the same site qualified at 95.

Limits

• The score weights are our design, not validated.
• Three runs per side is a small sample, and Kane varied from 14 to 24 steps on the same page.
• The demo sites have planted friction drawn from published agent-failure patterns.
• Human steps and Kane steps are different units.

What's next

Measuring against real people. A hand-counted optimum works for a demo page. On a live site, PostHog autocapture can count real visitors' clicks, field changes and submits on the booking form and how long they take, which turns the Agent Tax into a comparison with real humans on that page. It's built behind a flag and tested against stubbed responses. Running it on a live project is the next step.

Try it

• Repo, with every Kane run: https://github.com/shrithik2007/agent-lighthouse
• Live report: https://shrithik2007.github.io/agent-lighthouse/dashboard/

Built by Team Lighthouse at Kane CLI Hack Day SF. Questions about the scoring or the repair loop are welcome in the comments.