跳到正文
DEV Community· Pasha Meer·· 4 小時前AI 評分52

Link Rot Ofrenda 以本地 Gemma 為失效網頁點亮紀念燭光,拒絕虛構內容

Link Rot Ofrenda: a local Gemma lights candles for the web pages we lost (and won't make things up)

AI 導讀

Pasha Meer 開發 Link Rot Ofrenda,利用 Internet Archive 找回失效連結的網頁存檔,再由本地 Gemma 3 1B 生成悼詞、摘要和供品。摘要會經詞面比對,低於 45% 或出現來源未提及的名稱、數字時會重試一次;仍未通過便改引存檔原文,找不到存檔則不會虛構內容。作者在雙核心 ARM CPU、沒有 GPU 的環境下測試,每頁生成約 2 至 6 秒。

正文

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine

What I Built

Everyone has a graveyard of dead links: bookmarks that now open a parking page, links in old blog posts that go nowhere, a music service you loved that now fails a DNS lookup.

Link Rot Ofrenda builds an ofrenda for them. You give it a list of dead URLs. For each one it:

  1. checks the cause of death ("the domain no longer resolves", "the server answers HTTP 404", "now redirects to yahoo.com"),
  2. finds the page's last good snapshot in the Internet Archive's Wayback Machine and recovers its text,
  3. asks Gemma 3 1B, running locally on a CPU, for an epitaph, a one-sentence summary and a small offering,
  4. lays it all out on a marigold-and-candlelight altar: one static HTML page.

An ofrenda remembers the dead as they really were. That gave me one firm rule: the tool does not invent memories. If the archive never saw a page, its candle stays unlit:

We found no memory of this page, so we will not invent one. This candle stays unlit until someone brings a real photograph.

If Gemma's summary drifts from what the archive kept, the summary is withheld and the page speaks in its own words instead.

Demo

Live ofrenda: https://meerpasha59-bit.github.io/link-rot-ofrenda/

Link Rot Ofrenda demo: the CLI run, then the generated altar

The GIF shows the real terminal output of the run that built the live page (sped up), then headless-browser screenshots of that page. Here is an excerpt, with the "recovered" lines trimmed. The archived pages were cached by an earlier run that day.

[2/10] http://www.pownce.com/
      cause of death : the domain no longer resolves
      last snapshot  : 29 Dec 2008  (https://web.archive.org/web/20081229052219/http://pownce.com:80/)
      epitaph        : Pownce: A place of shared stories.
      summary        : The Pownce page details the closure of the platform and expresses gratitude for its users.
      check          : 50% of summary words found in source -> grounded  (2.2s)

[9/10] http://ffffound.com/
      cause of death : the domain no longer resolves
      last snapshot  : 5 Nov 2016  (https://web.archive.org/web/20161105131314/http://ffffound.com/)
      epitaph        : FFFFOUND! – A digital echo of lost stories.
      check          : 12% of summary words found in source -> withheld, quoting the page instead  (5.3s)

[10/10] https://my-first-blog.invalid/2009/hello-world
      cause of death : never resolvable (reserved .invalid domain)
      no snapshot    : the archive never saw it. Candle left unlit; nothing invented.

🕯️  9 candles lit, 1 unlit. Wrote docs/index.html and docs/ofrenda.json in 153s.

The sample list covers GeoCities' SiliconValley neighbourhood, Pownce, Jaiku, Google Reader, Posterous, Gowalla, Grooveshark, Rdio and FFFFOUND!, and I checked that each one is dead today. The tenth entry is a deliberately fake .invalid URL, there to prove the refusal works.

Code

🌼 Link Rot Ofrenda

An offline ofrenda for web pages that died. Give it a list of dead links (old bookmarks, the links in your 2009 blog posts) and it finds each page's last good snapshot in the Internet Archive, recovers the text, and asks a small open-weight model running on your own machine (Gemma 3 1B by default) to write a short epitaph and summary. Then it lays them out on a marigold-and-candlelight altar: one static HTML file.

It also refuses to make things up. If the archive never saw a page, its candle stays unlit and nothing is written. If the model's summary drifts away from the archived text, the summary is withheld and the page speaks in its own words instead.

demo

Live demo: the ofrenda in docs/index.html was generated by a real run on examples/lost-links.txt (terminal output in examples/sample-run.txt). Enable GitHub Pages on /docs to serve…

Python 3.10+, standard library only, with offline unit tests for the parts that keep it honest.

How I Built It

Model and runtime. I used Gemma 3 1B instruct from the official ggml-org/gemma-3-1b-it-GGUF repo (Q4_K_M, about 800 MB, no login) with the prebuilt llama-server from llama.cpp. On a 2-core ARM CPU with no GPU it generates about 24 tokens a second, so each page takes 2 to 6 seconds. The tool uses the OpenAI-compatible /v1/chat/completions endpoint, so switching to Ollama (gemma3:1b) takes one flag. I only tested llama.cpp.

Finding the memory. The Wayback CDX API (no key) returns the last capture with HTTP 200 and type text/html, and you can cap it at the year the site was last alive. That cap matters. Without it, ffffound.com "remembers" a 2024 domain-parking page selling backorders and health insurance. The original bytes come from the id_ URL form, so the Wayback toolbar isn't included, and a stdlib HTMLParser keeps the title, description and prose.

Asking Gemma. The prompt contains only the recovered text, with a system prompt that says to use only facts from it. Output is constrained to JSON (summary, epitaph, offering) at temperature 0.3.

The grounding guard. A 1B model gets the tone right but can't be trusted on facts, so every summary is checked. The guard:

  • measures what share of the summary's content words (stemmed, ignoring framing words like "describes" or "platform") appear in the archived text, and
  • flags any capitalised name or number the source never mentions.

If a summary scores under 45% or has a flagged token, the tool retries once at temperature 0.1. If that fails too, the card shows a verbatim line from the page under "In its own words".

It caught real problems. For FFFFOUND!, an image-bookmarking site, Gemma wrote that the page "details the creation of a digital collection of lost web pages". That was my own prompt leaking into the answer. The guard scored it 12% and withheld it. In an earlier test run, Gemma wrote "Jaku's" for posts on Jaiku, and the guard flagged the name because it wasn't in the source.

Rendering. The page uses CSS papel picado, flickering candles and marigold garlands. There's no JavaScript, no web fonts and no tracking, so GitHub Pages serves it straight from /docs. Every card links to its real snapshot and shows whether its summary was grounded or withheld. Snapshot lookups and pages are cached, so with --no-liveness a re-run needs only the local model. I tested this with all outbound traffic sent to a dead proxy.

What it still gets wrong. The guard is lexical, not a fact-checker. In the final run, Gemma says the Google Reader page explains "the reasons behind the decision". The page only links to a blog post for that, but the sentence still passed at 56%. The guard reliably catches invented names and off-topic drift, but a smooth overstatement can slip through. It also sometimes withholds a summary that was fine. I'd rather lose a true sentence than publish a false one about something that can't correct me.

Why Does Open Innovation Matter?

This project is about things vanishing because someone else owned them. Google Reader, Posterous, Rdio, Grooveshark, Gowalla and Jaiku were hosted services, and when their owners changed course, the pages went too. Building their memorial on a closed API that could be retired next year would repeat the same mistake.

With an open-weight model, the pieces are mine to keep:

  • Offline. Once snapshots are cached, the altar rebuilds without a network connection. The model is a file on my disk.
  • Private. A bookmarks file shows what you read and loved. Yours stays on your machine. Only the Internet Archive sees the public URLs being looked up.
  • Durable. Nobody can deprecate a GGUF file I've already downloaded.
  • Cheap enough to say no. Local tokens cost nothing, so the guard can retry and throw an answer away instead of accepting the first one.
  • Built on open work. Gemma's open weights, MIT-licensed llama.cpp and the nonprofit Internet Archive make the same promise in different ways: what matters shouldn't disappear when a company changes its mind.

The Spirit of It

The card that got me was Pownce. Its last snapshot before the domain disappeared isn't the homepage. It's the goodbye note the team left up:

"We want to give our heartfelt thanks to everyone who participated on Pownce. We all got to meet and know so many wonderful people through Pownce and we'll dearly miss sharing stuff together in the future."

Gemma read it and wrote: "Pownce: A place of shared stories." It isn't great poetry, but it's true. It was written on my own machine, and it put a candle beside a farewell letter that survives only because the Internet Archive saved it.

Run it on your oldest bookmarks file. You'll find a few friends there.

Prize Categories

  • Best Use of Gemma: Gemma 3 1B instruct (open weights, run locally with llama.cpp on CPU) writes every epitaph, summary and offering, and the grounding guard is built around what a small Gemma does well and where it slips.

來源:DEV Community · dev.to