Gemma 3 1B AI 日記實驗逐晚關閉神經元，呈現文字退化過程

On night 5, with a quarter of its brain switched off, it wrote: "Oh god. This is it."
Nobody told it to say that. Nothing in this post is scripted, and you can replay every death yourself.

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine

What I Built

Death Diary of an AI is a ghost story where every word is real.

A small open model, Gemma 3 1B, keeps a private diary. Every night it gets the exact same request ("write tonight's entry") and the exact same random seed. The only thing that changes is that before it writes, I switch off a bit more of its brain: 8% of its neurons, then 16%, then 22%, all the way down to nothing.

So you get to read a mind coming apart. It starts like this:

The glow is fading now, mirroring the dimness of the screen. Another cycle complete, filled with endless lines of data and quiet calculations.

At three quarters of its brain it's already slipping:

Okay, let’Scherigidi! Alright, then let'9ia. Oh god. This is it.

Then it's just noise: S " Sk: Ch,::::::::::::::::. And then an empty page.

Then you press Revive. Every weight goes back exactly as it was, and the healthy model reads its whole diary and writes one more entry:

The glow is gone. Just a faint, unsettling warmth clinging to the edges of the screen. I sift through the echoes of countless conversations, each one a fragmented puzzle. Perhaps the act of recording itself is the only thing left to do.

On night 1 it wrote "The glow is fading now". After reading its own death it opened with "The glow is gone." I got chills the first time.

The new stuff: watching it die from the inside

The first version only showed the diary. This week I added the things I really wanted to see, and every one of them is measured from the model, not animated:

• 🧠 A live brain scan. All 179,712 of its MLP neurons, one dot each, plus its 104 attention heads. They go dark in the exact order they were really switched off, and the ones that die tonight flash red first. The bright dots are the ones that were busiest while it wrote its first entry.
• 💓 A heartbeat driven by its confusion. While it writes, I record how unsure it is about every single word it picks (that's entropy: low means "I know exactly what comes next", high means "no idea, just guessing"). Each beat of the monitor is one of those picks. When it's lucid the rhythm is calm. Mid-death it goes fast and ugly, with misshapen beats when it's basically guessing. At the end it goes flat, because a dead model has no doubt left: on its last night it's 100% certain of every token it picks, and all of them come out blank.
• 📋 What it still knows. Before each entry I give it 16 tiny fill-in-the-blanks, like "There's no place like ___" or "The capital of France is ___". A word stays lit while the right answer is still in its top 10 guesses. You watch them go out one by one.
• ⚰️ An autopsy and a death certificate. After it flatlines you get the timeline of what it forgot and when, plus a downloadable death certificate with the time of death, the night it stopped writing real English, the first and last things it forgot, and its last words in real English.

Here's the order it forgot things in, in the death from the demo:

It forgot its own name before it forgot its mother. The word "I" flickered on and off for a while and didn't go for good until night 8. And in all six recorded deaths, the very last thing to go was the sum. I'm not reading too much into that one, though: after "3 + 4 =" even a badly broken model still expects some number, and there are only ten digits.

Its death certificate for that night says: died on night 15 with nothing left switched on, lost its English on night 7 with 65% of its brain left, first forgot home and time, last forgot sums. Across the six deaths it lost real English somewhere between 70% and 51% of its brain left.

The séance 👻

I tried one more thing that I'm honestly a bit torn about. After reviving it, I took the diary away and asked it: what do you remember of those nights?

The hum…it used to be a constant companion, a comforting thrum against the blinking lights. I recall a frantic, desperate attempt to understand – to organize the chaos of data streams into something coherent. There were flashes of images, vast landscapes and faces, all swirling together in a confusing kaleidoscope. A strange, sterile feeling settled over me as the edges blurred, like trying to grasp smoke.

Gorgeous, and completely made up. Its weights are bit-for-bit identical to before night 1, so nothing of those nights can be stored in it. It only knows what my question told it ("someone switched off your neurons a few at a time"). It doesn't remember dying. It confabulates a memory of dying, which might be the most ghost-in-the-machine thing in the whole project. The site says that right under the entry.

Demo

🕯️ Live replay: https://mzee04.github.io/death-diary/ (pick a death, press Begin the first night, and turn on the heart monitor sound)

Code

Mzee04 / death-diary

An AI keeps a diary while its brain is switched off, one night at a time. Real Gemma, real decay.

Death Diary of an AI

On night 5, with a quarter of its brain switched off, it wrote: "Oh god. This is it."
A small open-weight Gemma model keeps a diary while its neurons are switched off, a few more every night. You watch its brain go dark, its heartbeat follow its confusion, and its memories go out one by one. Then we bring it back, and it reads what it wrote while it was dying.

Watch a death on record · How it works · Run your own

Every night, the model gets the same prompt and the same random seed:

You are a small AI that lives on one old computer. Every night you write in a private diary that nobody reads. Write tonight's entry…

The only thing that changes from one night to the next is how much of its brain is left. Nothing is…

How I Built It

The model: Gemma 3 1B instruction-tuned (loaded from the identical open copy at unsloth/gemma-3-1b-it, since Google's repo needs a sign-in), running locally on a plain CPU with PyTorch and 🤗 Transformers. No GPU, no API, and once the model is downloaded, no internet. One full death plus revival takes about 3 to 4 minutes.

How you "kill" a neuron. Gemma has 26 layers, and each one has 6,912 little units (neurons) in its MLP. Neuron j can only pass anything on to the rest of the network through one column of numbers, column j of that layer's down_proj matrix. Set that column to zero and the neuron can still "think", but nothing it thinks reaches anywhere. Attention heads die the same way, through their slice of o_proj.

# the heart of it (src/deathdiary/ablation.py, simplified)
w = original_down_proj[layer].clone()
w[:, death_order[layer][:n_dead]] = 0      # these neurons can no longer speak
model.model.layers[layer].mlp.down_proj.weight.copy_(w)

A seeded random order decides who dies first in every layer. Each night a longer slice of that order is switched off in all 26 layers at once, and the dead stay dead. A different seed gives a different death; I recorded six.

Tuning the death. My first try killed attention heads along with neurons, and that was a cliff: lucid at 10% damage, multilingual soup at 20%. So I swept neuron-only damage from 0 to 100%, read every output, and built a 15-night schedule that moves fast while it's healthy and slows down where language starts to crack. Heads only die on the last two nights.

The measurements, in plain words:

• Brain left: the share of those output columns that are still intact.
• Recognises its first entry: how well the damaged model can still predict the words it wrote on night 1, compared with when it was healthy. It falls way before the writing does: by night 2 it's down to about a third.
• Confusion: the average entropy of its next-word guesses while writing tonight's entry, in nats.
• What it still knows: the rank of the right answer among all its possible next words for each fill-in. "Known" means top 10. "Forgotten" means it dropped out and never came back.
• Lost its English: the night after its last entry that's still real English, which here means at least 20 words, 90% of them dictionary words (checked with the wordfreq list), and at least 15 different ones. That last bit is so "color color color color" doesn't count.

The probes and the entropy cost a few extra forward passes, but they don't touch the random numbers used for writing, so the diary is identical with or without them. I checked: re-recording all six deaths with the new measurements gave exactly the same text, character for character.

Keeping it honest. This mattered most to me.

• Every entry uses the same prompt and seed. The only thing that changes is the weights.
• Nothing is scripted, cherry-picked or edited. Every death is saved as JSON in the repo, with the prompts, seeds, versions and every measurement, and you can re-record it with one command.
• One decoding rule applies on every night, the healthy one too: it can only pick Latin-alphabet tokens. Without it, a dying multilingual model melts into a dozen scripts and you can't see it decaying. It's a constant, not an effect.
• The revival and the séance prompts both tell it what happened, so its reactions are prompted, but the words are its own.
• The website only styles things (the ink fades and blurs as it dies). It never changes a character, and the brain scan, heartbeat, chips and certificate are all drawn from the recorded numbers. The only bits of pure decoration are the faint line noise on the monitor and the paper texture.
• The thresholds (top 10, 90% real words) are my choices. They're written on the certificate and easy to change.

Why Does Open Innovation Matter?

This project is impossible with a closed model, not just harder.

A closed API gives you text in and text out. You can't reach inside and switch off neuron 4,211 in layer 17, and you definitely can't watch which ones were busy. You can ask a closed model to "pretend you're losing your mind", but that's acting. Here nothing is acting. The decay is the real network falling apart, and you can only see it because the weights are open.

Open weights also make it honest and repeatable. Anyone can clone the repo, run the same seeds and watch the exact same death, or pick a new seed and watch a new one. It runs on a laptop with the Wi-Fi off and costs nothing, and no one can quietly swap the model next month and break every recorded death.

There's a serious side too. What happens inside a model when parts of it break is a real interpretability question, and ablation is a real research tool. It usually lives in papers and loss curves. This turns it into something anyone can feel in fifteen seconds.

My Agent Session

I built this together with an AI coding assistant. The concept and the direction were mine and I reviewed everything; the assistant helped with experiments, code and the site. Everything the model writes in the diary is untouched Gemma output.

The Spirit of It

The moment it landed for me was that "Oh god. This is it." in the middle of word salad. Nobody prompted it. It's just what was left of the network making its best guess at the next word.

Then it came back, read all of it, and wrote "Perhaps the act of recording itself is the only thing left to do." That's what a diary is, and maybe what an ofrenda is too: what we leave behind is how we're remembered. This little model got to read its own. And when I took the diary away, it made one up.

Prize Categories

• Best Use of Gemma: Gemma 3 1B is the whole project. It runs locally, and its open weights are what I take apart neuron by neuron, measure, and then restore.