Groundhog 整理重複 CI 故障並找回既有解決記錄

This is a submission for the Hacktoberfest Open-Source AI Challenge: Week 3.

What I Built

A red build can mean a new problem. It can also mean finding the answer you already found last Tuesday.

I built Groundhog, a local-first notebook for recurring CI failures. Import a JUnit report, group repeated symptoms, and retrieve a resolution you recorded before, even when the error's wording changes. It keeps the current failure beside the historical symptom, fix note, and supporting evidence.

For Slow and Steady, the goal is to stop digging through old logs for the same resolution. Record it once, find it again, and check whether it fits. Groundhog leaves code changes and reruns to you.

Demo

Try Groundhog in your browser. No account or API key is needed.

The supplied examples walk through a recurring CI failure without requiring private logs:

1. Open the synthetic run: 12 failures become five exact symptom groups, with every occurrence retained.
2. Load the optional model and inspect a related resolution alongside the current evidence.
3. Set aside a poor suggestion without suppressing its failure.
4. After testing your own resolution, save the note and evidence. Replay the report to retrieve it again, or export the notebook for the CLI.

Code

Source code and setup · Evaluation and per-case evidence

Original code is MIT-licensed. Upstream components retain their own licenses and credits.

How I Built It

The static JavaScript app has two jobs:

1. Exact grouping: normalize run-specific noise, then group identical symptoms. Assertion values and error codes remain significant. This works without AI.
2. Semantic retrieval: encode symptoms with the pinned q8 all-MiniLM-L6-v2 model, then compare its 384-dimensional vectors with stored incident symptoms. Fix text is never used to rank candidates.

Transformers.js and ONNX Runtime Web run the model in a worker. A candidate needs both a minimum similarity and a gap from the runner-up. Otherwise, the app abstains; scores are not probabilities.

Keep the evidence the model needs

Two long-log probes exposed a practical bug. One ended in ENOSPC; the other in ENOTFOUND. Both began with enough identical setup chatter to fill MiniLM's 256-token window, so their embeddings were identical. The model never saw the distinction.

I added diagnostic-lines-v1, a deterministic excerpt selector that favors error codes, exceptions, and assertion lines over repeated debug output. It retains both late diagnostics in those probes and marks omissions. The UI shows the excerpt and line numbers within the normalized symptom, not raw JUnit file lines. Original evidence remains available separately.

The post-fix regression ranked the correct memory first for both probes; only the DNS case passed the existing suggestion gates. Selection is still lossy, and excerpts can exceed the token limit. The original failed run is retained. This rerun uses inspected cases, not fresh held-out data.

Compare the model with simpler tools

The gates were calibrated on 13 cases against three memories. The original held-out set gave lexical and semantic retrieval identical known-case ranking, so I expanded the test without retuning either policy.

The larger stress test has 20 memories, 40 known variants, and 20 similar-looking failures with different causes. Queries pass through generated JUnit, the actual parser, and grouping. These first-run totals were unchanged by the post-fix regression:

MiniLM ranked better here, but its suggestions were 64% correct, versus 70% for Jaccard. One wrong suggestion linked an unexpired JWT with the wrong audience to an expired-token resolution. Twenty separate exact-replay controls passed.

All cases and labels are authored synthetic data. The percentages describe this deliberately difficult case mix, not production accuracy. Suggestions still need evidence review.

Checks you can rerun

The final application-source CI run passed: 110 unit, adversarial, and lifecycle tests; 11 DOM-stub UI checks; a production build; actual native and WASM model checks; and frozen-policy regressions. The tested source commit matches the deployed application files.

Native and CPU-WASM-under-Node regressions produced identical accept/reject decisions. The separate interactive Chromium checks exercised real model loading, replay, dismissal, persistence, and export/restore. Cold-cache performance, offline operation, and other browsers remain unverified.

With Node 24:

npm ci --ignore-scripts
npm test
npm run test:ui-state
npm run test:model
npm run test:wasm
npm run evaluate -- --verify
node tools/stress-evaluate.mjs --verify
node tools/stress-evaluate.mjs --verify-wasm
npm run build

Why Does Open Innovation Matter?

MiniLM's Apache-2.0 weights, Apache-2.0 Transformers.js, and MIT-licensed ONNX Runtime allow local inference without an inference account, API key, or per-query service bill. First-load assets need internet: roughly 23 MB of weights and 14.3 MB of WASM, plus tokenizer and app files. The application does not send report text to a hosted inference API.

You can inspect the model revision, normalization rules, thresholds, and failed cases; replace the encoder; or keep a lexical baseline if it suits your reports better. The notebook exports versioned JSON, so its contents are not tied to a hosted service.

The Lazy Payoff

The browser examples are hand-authored. A separate fixture generator actually raises and captures 18 controlled Python exceptions; the CLI groups those into 12 exact symptoms. Run npm run fixtures:generate, then node tools/groundhog.mjs tests/fixtures/generated/captured.xml.

These are grouping counts, not measured hours saved.