我試圖改良 InkMind 詩歌生成器，並選擇了 GPT-2

In our Transformers week, my group built a small poem-writing AI called InkMind. We trained it from scratch in Google Colab on about 13,800 poems from the Poetry Foundation. It reads text one character at a time, looking at 256 characters at once.

I learned a lot building it. But I’ll be honest: the poems aren’t good. Because InkMind only sees single letters and learned from a small amount of data, it spells words wrong, repeats itself, and forgets what it was saying. So for this assignment I asked: what existing model could do better?

I looked at three.

GPT-2 is a small version of the model family behind ChatGPT, with 124 million parameters. It’s built the same way as ours, but it learned from a huge amount of text, and it reads words in chunks instead of single letters.

Qwen3-4B-Instruct is a newer open model that follows instructions well. It would probably write prettier poems, but it needs about 8 GB of GPU memory just to load, so it’s slow and heavy on free Colab.

BERT is great at understanding text, but it was made to fill in missing words, not to write new ones. Wrong tool for poems.

I picked GPT-2, for three reasons.

First, it works like InkMind. Since we built our own, we can compare the two and see what really changes: the size, the way text is split up, and the training data.

Second, it fixes our biggest problem. GPT-2 keeps common words whole, so it doesn’t need to learn spelling from nothing like we did.

Third, it’s easy to run. It works on free Colab, so we can give both models the same prompts, like “The moon shines over the hills” or “In the heart of Kigali”, and compare the poems side by side.

Qwen3 would likely write better poems, but it’s much bigger and harder to understand from the inside. For a learning project, I’d rather know why the poems differ than just get nicer ones. If I only wanted the best poems, Qwen3 would be my next try.

GPT-2 isn’t perfect either. It wasn’t trained on poetry, so it can still ramble or repeat itself, and it can pick up biases from the internet. Bigger also doesn’t mean better by default, so I’ll only say it’s an improvement if our tests show it. After that, I’d like to fine-tune GPT-2 on the Poetry Foundation poems so it learns the style of real poets.

So that’s my choice: GPT-2 for InkMind. Next, I’ll run both models on the same five prompts and see what happens.