Jev AI 接入三款遊戲的實測：兩次失敗，一次成功

Code on GitHub: LAMDiceBot, deguri, CrazyArcadeCopy. Live apps at lamdice.com and deguri.ai.kr.
More posts at radarlog.kr.

I added AI to my bot and it got worse.

Kill rate dropped from 67% to 53%. 2,840 calls, 1 failure, $0.054 total. It cost almost nothing. It also did nothing.

That was failure number two. Failure number one was an AI-driven ball that wedged itself against a wall and never came down. Only on the third try did I find a spot where it actually works.

This is the log of wiring the same model into three games.

Why Jev

It started on Threads, Meta's text app. A post titled "wiring Jev AI into a game" got 725 views. "Game AI with Jev" got 445 in two hours. Posts that only showed game features couldn't break 100.

People weren't reacting to features. They were reacting to watching an AI make a call on screen.

Jev is a model from a company called TypeSafe, and it's not an LLM that writes. You send a state plus questions with fixed types, and you get back a probability distribution over your own options. No explanations, no code, no prose.

choice asks "which one", score asks "which level", noul asks "is this true". Nothing to parse. Your code reads it straight into an if.

curl -s https://api.typesafe.ai/v1/systemone \
  -H "Authorization: Bearer $TYPESAFE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"jev-latest",
       "state":"Help! My payouts have been failing for 3 days.",
       "questions":{"is_urgent":{"type":"noul","instructions":"Does this convey urgency?"}}}'

What comes back is one line: "noul": 0.95. A 95% chance it's urgent.

Price sealed it. $0.042 per million input tokens, output free. A call is a fraction of a cent, so a game server can ask hundreds of times per round without caring. Official latency is "about 100ms". I measured 205 to 285ms.

To a game programmer this looked like one more branch in a state machine. So I started with the most real-time thing I had. That's where failure number one came from.

Try 1: Time-attack bot — the ball got stuck on a wall

Deguri is my pixel-animal racing game, think a marble run with animals. The standalone build has a time-attack mode.

Your ball rolls down a hill, you only control tilt and jump, and an eagle snatches you if you're slow.

I put Jev in the pilot seat. Every 120ms I sent the state as a few English sentences and asked for one of six tilt options (choice) and whether to jump (noul).

First run, the ball glued itself to the right wall and stayed there.

The cause was the option text. Tilt means different things while rolling and while walking, and the walking description "fastest in the corridor" had leaked into the rolling options.

Jev reads literally. It saw "fastest" and picked hard_right every time.

// Before — walking description leaked into rolling options
hard_right: "fastest in the corridor"

// After — split by mode, facts only
rolling.hard_right: "tilt hard toward the right wall"
walking.hard_right: "walk right along the corridor"

Evaluative words like "fastest" drag the choice. Splitting the descriptions by mode and adding "barely moved down in the last 2 seconds (probably stuck)" to the state finally got it off the wall.

After the fix: 51.7 seconds, averaged over 5 seeds. Doing nothing at all: 58.1 seconds. A one-line rule that only jumps when the eagle comes: 44.3 seconds.

The AI lost to a one-line rule. And the eagle caught it every single run.

Requests went out every 120ms, answers took 205ms. By the time an answer arrived, the situation had moved on. On top of that, Jev is weak at timing numbers. It jumped 1.2 to 0.85 seconds before the eagle's grab and got stuck in cooldown.

So I changed the job. Code rolls every tilt × jump candidate 1.5 seconds ahead and writes the outcomes in words. Jev only picks.

candidate A: reaches the next section, eagle still far away
candidate B: hits the right wall and nearly stops
candidate C: jumps early, lands in the mud

18.56 seconds. Dodged the eagle every time. Picking from the same candidates with a code score: 20.86 seconds. Best human on the live leaderboard: 31.1 seconds, out of 9 players. Cost per run about a cent, because it asks hundreds of times.

What crushed the leaderboard wasn't Jev. It was the lookahead. Jev beat the code score by a little over two seconds. Evaluating candidates works. Piloting doesn't.

Try 2: Bomberman-style tactics — the placebo almost fooled me

The second game is my clone of Crazy Arcade, a Korean Bomberman-style game where you trap opponents with water balloons. Its bot redraws a danger map every 0.1 seconds and is already pretty good.

I gave Jev tactics only. Every 1.5 seconds it got the situation in sentences and picked one of hunt, item, or farm. State capped at 2,000 characters, each option at 200.

On design day I had no API key, so Jev was never called. Yet the "code-score picks" group was winning 80% of matches.

Digging in: delaying the bot's first decision by just 7ms moved the baseline bot's win rate from 1.7% to 38.3%. At 14ms it was 93.3%.

I still don't know why. What was clear is that a few milliseconds of timing, not tactics, were swinging the results.

If you don't measure the placebo band first, you can't trust any number.

Four days later, with a key, the real run: 3 maps × 10 seeds. Kill rate against the expert bot went from 67% to 53%. 2,840 calls, 1 failure, median latency 285ms, 1.289M input tokens, $0.054.

67% → 53%
2,840 calls. $0.054. Down 14 points.

Minus 14 points sits inside the 10-point placebo band, so I can't say "definitely worse". But there's no evidence of better either. Jev kept choosing "item" out of the three and pulling the bot away from hunting.

Same reason as try one. Code sees a danger map every 0.1 seconds. Jev sees a few sentences every 1.5 seconds. The side with less information can't win. I'd stacked AI on top of a decision the code already made well.

So I moved it. Not a player, a coach. Every 30 seconds it sets difficulty up/keep/down, picks a bot personality with choice, and every 10 seconds hands down one game plan.

Tick-level decisions stay with the code. That's running in production now.

Try 3: Race highlights — it worked, but only picked the finish

The third try flipped direction. Not real time. A race that's already over.

When a Deguri race ends, the server builds candidate 6-second highlights: an overtake, an eagle snatch, a hole, mud, a nap, a scuffle, a neck-and-neck stretch.

For each one it gathers the facts as numbers, validates them, then rewrites them as English sentences. No player names, no room names, no coordinates. Just lane numbers, animals, and events.

Then one choice.

{
  "model": "jev-latest",
  "state": {
    "task": "Choose the best 6-second highlight of a finished race.",
    "race": "8 pixel animals rolled down a hill. The rabbit led most of the way...",
    "what_makes_it_fun": "The moment the outcome flips is the core. Everyone already saw the finish."
  },
  "questions": {
    "pick": {
      "type": "choice",
      "instructions": "Which clip would be the most fun to watch?",
      "criteria": {
        "c1": "the leader was snatched by the eagle",
        "c2": "nothing special happened"
      }
    }
  }
}

{
  "model": "jev-1.13.0",
  "answers": {
    "pick": {
      "type": "choice",
      "choice": "c1",
      "probabilities": { "c1": 0.88, "c2": 0.12 },
      "confidence": 0.76
    }
  },
  "usage": { "input_tokens": 318, "output_tokens": 34 }
}

It worked. The chosen scene shows up as a GIF on the result card.

Then I ran it 20 times and 18 picks were the finish.

The candidate text had location hints like "at the very end of the race". Same trap as "fastest" in try one.

// Before — location hint drags the pick
"the rabbit took the lead at the very end of the race"

// After — no location
"the rabbit overtook the leader after trailing for most of the race"

I stripped locations and made the code always include a "decisive moment" candidate. Since then it only picks the deciding scene in races that were actually close.

Testers said a close race is as fun as the deciding moment, so I added time spent neck and neck and the gap at the finish line to the facts.

As words, not numbers. neckSec: 4 became "stayed neck and neck for several seconds". Jev is bad at comparing numbers.

Reactions feed back in. 👍/👎 on the result card are aggregated per fact into a like ratio that scales the code score between 0.5× and 2×. And one race in five is picked by code score alone, so I keep checking whether Jev is really better.

What I'm taking to the next project

I collapsed the call code from all three repos into one function. No key, failure, timeout, or nonsense answer all return null. The caller falls back to code rules.

const JEV_TIMEOUT_MS = 3000;

async function askJev(state, questions) {
    const key = process.env.TYPESAFE_API_KEY;
    if (!key) return null;
    try {
        const res = await fetch('https://api.typesafe.ai/v1/systemone', {
            method: 'POST',
            headers: { Authorization: `Bearer ${key}`, 'Content-Type': 'application/json' },
            body: JSON.stringify({ model: 'jev-latest', state, questions }),
            signal: AbortSignal.timeout(JEV_TIMEOUT_MS)
        });
        const body = await res.json().catch(() => null);
        if (!res.ok || !body || !body.answers) return null;
        return body.answers;
    } catch (err) {
        console.warn('[jev] failed:', err && err.name);
        return null;
    }
}

const answers = await askJev(state, { pick });
const pick = answers && ['c1', 'c2'].includes(answers.pick.choice)
    ? answers.pick.choice
    : codeFallback();

Re-check that choice is one of the options you sent, and time out at 2 to 3.5 seconds. On a real-time path, no retries, straight to the fallback.

All three tries pointed at the same principles. Code owns the flow, Jev makes one narrow call. Code does the math and Jev gets words.

One question, one judgment, and bundle several questions into one request since they're evaluated in parallel, so latency stays flat and you only pay for tokens.

The key lives in a server environment variable only, and there are caps: one in-flight request per socket, 60 per minute, one per race for highlights. The "Jev" badge only shows when Jev actually answered.

And run the placebo group first. In a world where 7ms moves a win rate from 1.7% to 38.3%, a number without a control group isn't a number.

Every try came down to one question. Is the code already making this decision well? If so, don't add AI.

What code can't do is common-sense judgment like "which of these is more fun to watch", and that landed best when I asked about something that had already happened.

"Put the AI in the coach's seat, not the player's. Better yet, let it watch the replay."