AI Grand Prix 如何呈現小型決策模型的機率判斷，以及打造誠實 AI 示範的經驗

Five AI models, one race track, zero cloud

The demo problem

Every AI demo you have watched in the last two years looks the same. A prompt box, a streaming answer, a slow reveal. It is a good demo, and it has taught everyone the same lesson: AI means a model that writes.

But most of the models shipping onto devices this year are not writing anything. They are picking an option. A phone decides whether to surface a notification. A camera decides whether a frame is worth keeping. A car decides whether the object ahead is a shadow or a hazard. These are decision models: you hand them a state and a list of options, and they return a probability for each option in a single forward pass.

Nobody has seen one work. That is the gap this project fills.

AI Grand Prix puts five small decision models on the same track, in the same car, and lets you watch them drive.

What I built

Five open-weight models race on a closed circuit. Twenty times a second, each one is handed a JSON state describing its car and the road ahead, and returns a probability for each of five steering options. The car then drives the option drawn from that distribution.

Nothing is generated. There is no text, no prompt, no output parsing. The whole inference call is one HTTP POST to POST /v1/systemone and the answer is a set of probabilities.

The part worth watching is not the racing. It is the belief cone: a translucent fan drawn out of every car, one wedge per steering option, sized by the probability the model just returned. When a car runs wide on a corner, you watched it choose the less likely move.

Architecture

Four decisions shaped this architecture, and each one is a lesson that generalises past this project.

1. One WebGL context, not five

Five racer panels could have been five canvases. On a laptop that means five WebGL contexts competing for the same GPU, and it degrades badly. PixiJS gives us one renderer and one WebGL context, with each panel drawn as a masked viewport inside it.

I picked PixiJS rather than a game engine on purpose. PixiJS is a renderer, not a physics engine, and the interesting drawing here is custom: a probability fan, a steering gauge, a track ribbon. A game engine would have handed us a scene graph and then fought us on the parts I actually cared about.

2. Stdlib HTTP, not FastAPI

The gateway is Python's http.server. That is not a lack of ambition. The single most important property of a demo like this is that a fresh clone runs. No pip install, no virtualenv, no version conflict, no network. If a reader needs to install something before they see a car move, you have already lost most of them.

3. Decisions are off the render loop

This is the one that bites everybody building agent demos.

Our fastest racer answers in about 17 ms at the median. Our slowest measured decision on an M1 Pro was 861 ms. That is a 50x spread, and it comes from the same five models on the same machine.

If you tie a decision to a frame, the whole board stutters every time the slow model thinks. So we split the rates:

• physics and rendering at 60 Hz
• decisions at 20 Hz per racer
• a decision in flight never blocks a frame, and the car holds its last steering input until the next one lands

The panel shows p50 and the live decision rate, so the latency is visible instead of hidden behind a smooth animation.

4. Sampling, not argmax

This is the most interesting decision in the project, and it was a judgement call rather than a technical one.

The obvious implementation is argmax: take the highest probability and drive it. That produces a field of perfectly obedient cars that never make mistakes, and it hides the entire point of a decision model.

I sampled from the distribution instead:

def draw_action(probabilities, policy, seed, tick):
    """Pick one option. `sample` draws from the distribution; `argmax` takes the top."""
    if policy == "argmax":
        return max(probabilities.items(), key=lambda item: item[1])[0]
    ...

The sampling is seeded per racer and per tick, so a race is reproducible, and both the distribution and the drawn action are recorded in telemetry. The consequence on screen is that a model saying 45 percent hard left and 40 percent left will sometimes take the 40, clip the apex, and run wide.

That is a real uncertain decision, and it is the moment viewers understand what they are looking at. policy is a config value, so you can flip to argmax and watch the personality drain out of the field.

The state a model actually sees

There is no hidden context and no prompt history. This is the entire input, built in sim.py:

return {
    "task": "steer",
    "track": {
        "width_m": round(self.track.half_width * 2, 1),
        "bends_ahead": bends,
    },
    "car": {
        "lateral_m": round(car.lateral, 2),
        "side": "right of centre" if car.lateral > 0 else "left of centre",
        "heading_error_deg": round(math.degrees(car.heading_error), 1),
        "speed_mps": round(car.speed, 1),
        "lap": car.lap,
        "laps_total": self.laps,
        "off_track_ticks": car.off_ticks,
        "spins": car.spins,
        "position": place,
        "cars_total": len(self.cars),
        "gap_to_leader_m": round(max(0.0, leader.distance_m - car.distance_m), 1),
    },
    "goal": (...),
}

And the options are a fixed list, each with a description that is sent to the model as the option's criterion:

STEER_BUCKETS: tuple[tuple[str, float, str], ...] = (
    ("hard_left", -0.036, "Steer hard left: a firm left arc, about a 28 m radius."),
    ("left", -0.018, "Steer left: a gentle left arc, about a 55 m radius."),
    ("straight", 0.0, "Hold the wheel straight: no steering input."),
    ("right", 0.018, "Steer right: a gentle right arc, about a 55 m radius."),
    ("hard_right", 0.036, "Steer hard right: a firm right arc, about a 28 m radius."),
)

The request body, in full:

body = {
    "state": state,
    "questions": {
        question_id: {
            "type": "choice",
            "instructions": instructions,
            "criteria": {option.label: option.description for option in options},
        }
    },
}

One POST in, one set of probabilities out. No streaming, no tokens, no parsing.

Four bugs worth publishing

Most of the work in this project was not the model integration. It was making the simulation honest. Each of these produced a demo that looked correct and was lying.

The lap counter handed out free laps. Cars were placed just behind the start line, so the first physics tick crossed it and credited a lap after 13.5 m. A car "won" a 700 m circuit in half a second. The fix has two parts: the grid starts past the line, and a lap only counts after most of a circuit of real travel.

crossed_line = previous_s > 0.75 * self.track.length and self.track.s_at(index) < 0.25 * self.track.length
if crossed_line and car.since_lap_m >= LAP_MIN_TRAVEL * self.track.length:
    car.lap += 1
    car.since_lap_m = 0.0

Distance was taken modulo the circuit. A car that stepped backwards across the start line had its travel wrap to nearly a full lap. Three cars showed identical 1020 m on a 700 m track while all still on lap zero. Travel is now unwrapped, and a regression test drives a car backwards to prove it earns nothing.

The maps were undrivable and it looked like the models were broken. Generated circuits produced 1 m corners while the sharpest steering option turns on a 27.8 m radius. The cars spiralled, spun fifteen times a lap, and it read as "these models are bad." The models were fine; the tracks were impossible. There is now a tool that fails the build if any corner is tighter than the sharpest option:

python3 tools/diagnose_track.py

The canvas was never attached. The user reported a blank board while the race ran. The DOM was perfect, there were zero page errors, and PixiJS had created its canvas. It simply had not been appended to the document. PixiJS v8 does not attach it for you. The fix is one line, and the lesson is that "the DOM looks right" is not evidence that anything rendered.

That last one is worth repeating: a screenshot from a vision model told us the gauges were missing when they were present, and the DOM said the canvas existed when it did not. The only reliable check was reading the live scene graph out of the running page.

Honesty rules I held to

1. An unavailable engine freezes its car. It is never asked to guess and it never answers on another model's behalf. The panel greys out and prints the reason.
2. Latency is measured, not estimated. tools/probe_engines.py prints the distribution each engine actually returned, and the panel shows p50 from the live race.
3. This is a showcase, not a benchmark. Same physics, same options, same seed, one narrow task. Nothing here claims a model is better at driving in general.
4. No model weights in the repo. They are downloaded into your own cache, and every one of them keeps its publisher's license.

What the models look like in practice

The spread is the story. A 144M model answering in 18 ms and a 9B model answering in a second are both "a decision model", and picking between them for an on-device feature is exactly the trade this board makes visible.

What I would add next

The steering options are just a list of choices, and the simulator is just the consequence. Swapping both gives you a different game with the same belief cone, the same telemetry and the same tests: a drone landing pad, a trading bot, a traffic simulator.

Closer in, the three things I would build next are a lap time sparkline under each panel (all the data is already in telemetry), a per-racer top speed so latency becomes a visible handicap, and a decision log view that shows state in, distribution out, option taken, latency, for the last twenty decisions. That last one is the teaching view, and it is the reason the telemetry records distributions rather than just labels.

Run it

python3 tests/run_all.py     # 35 tests, no models, no network
./run.sh --replay            # two synthetic racers, watch the board
./tools/engines_up.sh        # then bring up the real models
./run.sh                     # pick your racers and map, press Start

The code is MIT. The models keep their own licenses.

Code & more: https://www.dailybuild.xyz/project/284-grand-prix