logsloth 讀取失敗的 CI 日誌，整理錯誤位置與修復建議

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

What I Built

I'm Anand Biju. I built logsloth, a small command line tool for the most boring part of my week: a red CI run.

The red X is never the bad part. The bad part is opening the log. You get a few hundred lines of checkout, cache restore, dependency install and cleanup, and somewhere in the middle is the one line that matters. The last line, Process completed with exit code 1, tells you nothing. So you scroll, you search for "error", you hit forty false matches, and ten minutes later you find out a package was missing.

logsloth does that scrolling. Give it a log file, pipe a command into it, or point it at a failed GitHub Actions run:

logsloth ci.log
npm test 2>&1 | logsloth
logsloth --gh https://github.com/owner/repo/actions/runs/1234567890

It prints a short card with four lines. CULPRIT is the real error line. WHERE is the file and line. WHY is one sentence on what went wrong. FIX is the next thing to do. Below that are the handful of log lines it used as evidence, with their original line numbers, so you can check it instead of trusting it. --md prints the same thing as Markdown for a PR comment, and --json is there for scripts.

Demo

These are all real runs on my machine. The first three use real logs from public projects, pulled with gh run view --log-failed.

A ruff-format pre-commit failure in psf/requests. The log is 284 lines and the run ends with about forty lines of git cleanup after the error. logsloth skips that and lands on the hook that failed:

A mypy error in pallets/flask. The error message wraps over three lines, and logsloth joins them back together:

A failing vitest assertion in vitejs/vite. The log is 653 lines. Here Gemma got distracted by a pnpm store prune line in the excerpt and suggested running it. The validator threw that out, and the card shows the built-in hint instead. The footer says so.

And one of the bundled examples, a pytest run where httpx was never installed:

Code

Anandb71 / logsloth

Reads a failing CI log so you don't have to. Code finds the evidence, a local Gemma explains it.

logsloth

A small CLI that reads a long failing CI, test, or build log and tells you the one thing that broke.

Code finds the evidence and states the facts. A local open-weight model (gemma3:1b via Ollama) only explains why it failed and suggests a fix. The model never sees the whole log.

Why

When CI goes red you get a few hundred lines of setup, cache restores, and cleanup, with the real error somewhere in the middle. Finding it is boring and you do it many times a week. logsloth does the scrolling for you and runs entirely on your machine, so private logs never leave it.

Install

pip install -e .

You also need Ollama and the default model:

ollama pull gemma3:1b

OLLAMA_HOST defaults to http://127.0.0.1:11434.

Usage

# a log file
logsloth path/to/ci.log
# piped from a local command
npm test 2>&1 | logsloth

# a failed

…

ollama pull gemma3:1b
python3 -m venv .venv && . .venv/bin/activate
pip install -e .
logsloth examples/pytest_httpx.log

python -m pytest -q runs 47 tests. They never call the model or the network.

How I Built It

The rule I followed: code finds the facts, the model only explains them.

My first version let Gemma 3 1B write everything. It went badly. On a cargo error it wrote the literal text "one sentence: what broke" as the culprit. On a jest log it pointed at the test file instead of the file that was missing the import. On the real flask and requests logs it blamed git config and safe.directory, because the cleanup lines at the end of a GitHub log look like errors. That third one is why I stopped testing only on my own example logs.

So the work is split.

Triage is plain Python. It strips ANSI colours, including the literal ^[[36m that gh run view --log-failed leaves in, plus the timestamps and the job<TAB>step prefix on every line. It folds repeated lines and then scores lines against known failure patterns: Python tracebacks and pytest E lines, Cannot find module, rust error[E...] with its --> location, mypy and ruff style path:line: error:, pre-commit hooks that print Failed, pip's No matching distribution, docker's failed to solve, and a few more. The final exit code 1 line can never be the main finding if anything else matched. CULPRIT and WHERE come straight from this step, so they are always lines that really are in the log.

Gemma only writes why and fix. It gets a short excerpt, between 6 and 15 lines in the runs above, not the whole log. The prompt only includes the rules that apply to this failure. When I listed every rule at once, the 1B model copied them, and suggested "install httpx using pip or npm".

Every answer gets checked. Why and fix are validated separately. They have to be non-empty and short. They can't repeat the prompt or the example. They can't tell a git story when the failure isn't about git. They have to name the real package, symbol or hook. If the fix says to install something, that name has to appear in the log. That check caught Gemma telling me to "include lodoc" when the log said lodash. A field that fails is replaced by a built-in hint for that error type, and the footer says which parts came from the model and which from the hint, for example why from gemma3:1b, fix from hint.

In my final runs over the seven logs, Gemma's why was kept in six and its fix in three. On the vite log both fields fell back to hints. On this shared, CPU-only box a call took between 1.9 and 34 seconds, and most were around 10.

It is standard-library Python. Ollama serves the model over HTTP on localhost. I built it this week with the Cursor CLI agent doing a lot of the typing, while I wrote the spec, ran it against real logs, and kept tightening the validator.

Why Does Open Innovation Matter?

CI logs are full of things you shouldn't paste into a hosted chatbot: internal hostnames, paths, package names from a private registry, sometimes a token someone forgot to mask. With an open-weight model on my own machine, the log stays here. Nothing gets uploaded, there's no API key to manage, and running it a hundred times a day costs nothing.

It also changed how I built it. A 1B model running locally is fast and cheap enough to call on every failure, but it is wrong often enough that I had to treat its output as a suggestion to check, not an answer. Because I control the prompt, the temperature, the retry and the validation, I could build that checking around it. Swapping in a different model is one flag, --model.

The Lazy Payoff

Before: open the run, expand the failed step, scroll past setup, search for "error", skip the false matches, find the real line, then work out which file it means.

After: logsloth --gh <run url>. I read four lines, then check the evidence lines if I don't believe them. On the 653-line vite log the card had the failing spec file and line, and the expected and received values, in the evidence block. I didn't have to open the log.

Prize Categories

Best Use of Gemma. logsloth runs Gemma 3 1B (gemma3:1b) locally through Ollama. It writes the why and fix, and the validator decides which of its answers to keep.