Night Shift 以本地模型分析 Oracle 日誌，生成兩行晨間摘要

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

On Tuesday afternoon at 18:39 UTC, a background process in one of my lab databases died:

2026-10-20T18:39:10 Instance Critical Process (pid: 7, ospid: 3437, MMAN) died unexpectedly
                    PMON (ospid: 3425): terminating the instance due to error 822
2026-10-20T18:39:13 Instance terminated by PMON, pid = 3425

MMAN manages the database's memory. When it dies, Oracle takes the whole instance down with it, and that's what happened. Then nothing happened for five days. No alert, no email. The alert log doesn't write anything once the database is gone. I found out on Sunday evening, by accident, while looking for something else.

Every DBA knows the cure, and nobody actually does it: read the alert log every morning. It's three hundred lines a night of log switches, resizes and scheduler windows. Somewhere in there might be the one line that matters. Some mornings there isn't one, and the morning there is, you've stopped reading.

So I built a sloth to do it for me.

What I Built

Night Shift reads your Oracle alert and listener logs overnight and leaves you two lines in the morning. Everything else stays in a list underneath, in case you want it:

# Night Shift

> lab-26ai had an out of space issue; lab-se2 is down. lab-26ai also saw 37 checkpoint issues and 1 deadlock.
> Start lab-se2 immediately, then check the alert log entries from before it stopped.

## lab-26ai: up
2026-10-24 23:01 to 2026-10-25 23:01 UTC · 303 log lines, 263 routine

- 🔴 Out of space in a tablespace (ORA-01653), 2026-10-25 22:47 UTC. Add or extend a datafile, or find what grew.
  FREEPDB1(3):ORA-01653: unable to increase tablespace NS_TINY [FREEPDB1] by 1MB during insert or update on table LAB.NS_FILL
- 🟡 Checkpoint not complete, 37× 2026-10-25 01:00 UTC → 2026-10-25 21:06 UTC. Redo logs are too small or too few for the load.
- 🟡 Deadlocks (ORA-00060), 2026-10-25 22:47 UTC. Read the trace file: it names both sessions and the SQL.

## lab-se2: DOWN
- 🔴 Database is not running now. no pmon process for ORCLCDB; last alert log entry 2026-10-20 18:39 UTC

Night Shift · Temporal run night-20261025-230121 · 1 s · summary by gemma4:e4b

That's a real run from tonight, trimmed a little, against two lab databases on my desktop: Oracle 26ai Free and 12.2 Standard Edition 2. To give it something to find, I filled a 16 MB tablespace and caused a deadlock on the 26ai one. The SE2 one was still dead from Tuesday (more on that at the end).

It's set up once and then left alone. A Temporal Schedule runs it at 6 AM. The brief lands in a Markdown file and a desktop notification. The model runs locally in Ollama. Night Shift never connects to the database: it reads the log files and checks the process list, and nothing leaves the machine.

Demo

Code

pyaroslav / night-shift

Reads your Oracle alert and listener logs overnight and leaves a two-line brief for the morning. Gemma 4 on Ollama + Temporal, all local.

Night Shift

Night Shift reads your Oracle alert and listener logs overnight and leaves a two-line brief for the morning. It runs on your own machine: Gemma 4 on Ollama writes the brief, and a Temporal workflow makes sure it arrives.

I built it because my lab database died on a Tuesday afternoon and I found out on Sunday night.

# Night Shift
> lab-26ai has an out of space tablespace; lab-se2 is down; lab-offsite logs are unreadable.
> Start lab-se2 and review its logs from before it stopped.

## lab-26ai: up
2026-10-24 22:56 to 2026-10-25 22:56 UTC · 302 log lines, 262 routine

- 🔴 Out of space in a tablespace (ORA-01653), 2026-10-25 22:47 UTC. Add or extend a datafile, or find what grew.
- 🟡 Checkpoint not complete, 37× 2026-10-25 01:00 UTC → 2026-10-25 21:06 UTC. Redo logs are too small or too few for the load.
-

…

How I Built It

The obvious version of this project is one line of code: paste last night's alert log into a model and ask for a summary. I built that version too, as the baseline, and it's the reason the real one looks the way it does.

Rules sort the night, the model writes two lines

logs.py splits the alert log into timestamped entries. It handles 12c+ ISO timestamps and the old 11g Tue Oct 20 18:39:10 2026 style, plus the listener log. Each entry goes through about twenty plain patterns, written so a DBA can read the file and agree with every line:

• Critical: instance terminations, dead background processes, ORA-600/7445, block corruption, a stuck archiver, a full tablespace, missing online redo logs, ORA-4031, running out of processes, the listener losing the service.
• Look today: deadlocks, checkpoint not complete, ORA-1555, failed scheduler jobs, connection timeouts, failed logins, swapping.
• Routine: log switches, resizes, scheduler windows, service updates. These are counted, never reported.

Repeats are folded together. A job that fails every ten minutes is one finding with a count. One crash that writes four alert log entries ("terminating", two state dumps, "terminated") counts as one incident, not four. An ORA- code that matches no rule still shows up as "Other Oracle error", so a problem I didn't think of can't quietly disappear. And if you've decided to live with something, like the Statistics Advisor job that fails every ten minutes on a fresh 12.2 CDB, there's a mute list.

A deadlock line in the alert log only says "see the trace file", and nobody opens trace files at 7 AM. So for ORA-00060, Night Shift opens the trace itself. It reads the deadlock graph to tell the kind of deadlock apart: TX locks held and wanted in exclusive mode mean two sessions updated the same rows in opposite order; a TM lock usually means a foreign key without an index; share-mode waits point at duplicate-key inserts or bitmap indexes. Then it pulls each session's user and SQL. The client machine and program name stay out of the brief:

- 🟡 Deadlocks (ORA-00060), 2026-10-25 22:47 UTC. Row lock deadlock: two sessions updated the same rows
  in opposite order. Make the application touch rows in one order (e.g. by key), or commit sooner.
  - session 230 (LAB): update ns_dl set v = 2 where id = 1
  - session 42 (LAB): update ns_dl set v = 2 where id = 2

Two checks look at what isn't in the log. Is pmon running? And did the alert log write anything at all? A database that crashed on Tuesday writes nothing on Wednesday, so a silent night is a critical finding, not a quiet one. My first run of the pmon check told me lab-26ai was down. It wasn't: since 23ai, the process is called db_pmon_FREE, not ora_pmon_FREE. Both names are checked now.

Then Gemma 4 (e4b) gets the night as a small JSON digest (about 500 characters per database), not the log, and writes exactly two lines. Its answer is checked before anyone sees it:

• exactly two lines;
• every number in them has to appear in the digest;
• every database with a critical finding has to be named.

It gets three tries, and then a plain template writes the lines instead. The list under the two lines never comes from the model, so even a bad summary can't hide an outage.

Temporal makes sure it arrives

A morning brief that silently doesn't arrive is worse than no tool at all, because by then you've stopped checking yourself. That's why it's a Temporal workflow and not a cron job:

• Each database is read in its own activity, with retries at 15 s, 30 s and 60 s. A host that stays unreachable doesn't crash the run. It becomes a finding: "Couldn't read the logs. The night is unknown."
• If Ollama is down all night, the summary activity fails three times and the template writes the brief instead.
• If the worker dies halfway, the run picks up where it stopped as soon as a worker comes back. Steps that already finished aren't repeated.
• A Temporal Schedule starts it at 06:00 America/Chicago, with the overlap policy set to skip.

All of it runs locally. night-shift server starts Temporal's single-binary dev server with a SQLite file, so the history survives a reboot, and two systemd user units in the repo bring the server and worker back up afterwards. If the machine is asleep at 6 AM, the Schedule's catch-up window starts the missed run when it wakes up.

Two of those promises (an unreachable host, and no model all night) have a test on Temporal's time-skipping test server. It runs in about a second in CI. For the rest, I tested by hand: I added a third "database" whose container doesn't exist. I started a run, killed the worker with kill -9 twenty seconds in, and started it again thirty seconds later:

The two real databases were read in under a second. The missing one was retried four times over 1 m 45 s, across the restart. Then Gemma wrote the summary and the brief was delivered, with the missing host as the third critical finding. The first time this happened wasn't planned. My first worker crashed on startup: Temporal's workflow sandbox doesn't allow Path.resolve() at import time. The run I'd started just waited in the queue for ten minutes. When I fixed the worker, the run finished as if nothing had happened.

Does the brief say what matters?

I replayed 22 nights from the two lab databases' logs (Oct 13 to tonight), each one the 24 hours up to 6 AM Central. I compared Night Shift with the one-line version: the same night's raw alert log, the same model, the same two-line instructions. The events that must be in the two lines were found with plain grep on the raw log, independent of my rules: an instance died, a tablespace filled, a deadlock, a missing online redo log, or an alert log silent all night. That gave 8 events across 5 nights.

What the numbers hide:

• The raw log is good on small nights. On the night lab-se2 died, there were only four entries, and it wrote the better line: "Instance terminated at 18:39:13 due to MMAN dying unexpectedly, triggering PMON error 822." Night Shift's line said the instance terminated, but not why. The why is in the list underneath.
• On big nights it stops following instructions. Four nights didn't fit the 32K-token context I gave it, so Ollama cut them down to about 16K tokens and most of the night was lost. On all four it ignored "two lines" and wrote a multi-page essay with headings and a priority table. On one of them, the night's only deadlock was in the part that got cut, so the model never saw it.
• It can't count. On four different nights it told me how many log switches there had been: 54, 33, 87 and 45. The real numbers were 61, 69, 58 and 77. That's four for four wrong, and each one sounded completely sure of itself. The other three nights' numbers are rounded file sizes it converted from KB, values from SQL it wrote itself, and the next log sequence after the window. Night Shift's counts come from Python. The model only gets to repeat them, and the number check rejects anything it makes up.
• It can't see silence. The night after the crash, there was nothing to send it.
• Night Shift's miss: on lab-se2's first night, the two lines named the shutdown, the missing redo log and 17 job failures, but not the 2 deadlocks. They're in the list, but not in what you'd read at 7 AM.

The honest caveats:

• These are lab logs, and the must-report events are few.
• One of the two "instance died" nights is a shutdown abort I ran during setup.
• The tablespace was filled on purpose.
• I changed the prompt once after the first replay. The two lines kept skipping deadlocks, and Gemma was writing "as per the finding's 'do' field". That first run scored 6/8 and is in the repo next to the second. I also loosened one scoring pattern from "silent" to "silen" after Gemma wrote "alert log silence", which gave Night Shift one of its 7.
• A rerun with the final code, which adds the restart rule, scored 8/8 (results/replay-v3.json). The two lines vary a little from run to run, so the table keeps the lower number.

Why Does Open Innovation Matter?

An alert log is a map of someone's infrastructure. It has host names, file paths, user names, SQL fragments, and the exact times their systems fell over. A DBA can't paste a client's into a hosted chatbot. I wouldn't want to give a SaaS agent a login to production just to read it, either.

The open pieces are what make this possible:

• Gemma 4 on Ollama runs on the same desktop as the databases, at no cost per night, with no internet needed.
• Temporal is open source, so the durable part runs locally as one binary and a SQLite file. Nothing is hosted anywhere, and you still get retries, resume and schedules.
• I can see why the model said what it said. The exact digest it read is right there. When it's wrong, I fix a rule or the prompt instead of hoping the next model version is better. A closed API wouldn't have let me run the raw-log baseline on 22 nights for free either, and that baseline is what told me where the model needs help.

The Lazy Payoff

Reading two alert logs properly takes a good ten minutes on a busy night, and nothing at all on the nights you skip it, which is how I lost five days. Now it's two lines with my coffee, and the full list only when a line is red.

Its first real run told me what I'd missed all week: lab-se2 is DOWN, last alert log entry Tuesday 18:39 UTC. So I started it. The next brief had moved on by itself:

## lab-se2: up
2026-10-25 00:03 to 2026-10-26 00:03 UTC · 90 log lines, 85 routine

- 🟡 Instance started, 2026-10-26 00:02 UTC. Check the restart was planned; the entries just before it say why it stopped.

It also points at the listener's "address already in use" lines from the restart, which are worth a glance and nothing more. The Temporal Schedule fires for the first time at 6:00 tomorrow morning, and this is what will be waiting:

Tonight, the sloth is on shift.

Prize Categories

• Best Use of Temporal: the brief is a Temporal workflow with one activity per database, retry policies, and a fallback when the model is down. A host that can't be reached turns into a finding instead of a crash. A Temporal Schedule runs it at 6 AM. I tested it by killing the worker mid-run and pointing it at a host that doesn't exist (timeline above). Everything runs locally on Temporal's open-source dev server.
• Best Use of Gemma: Gemma 4 (e4b) runs locally in Ollama and writes the two lines from a compact digest, checked for invented numbers and missing databases. The replay compares it with the same model reading the raw log, night by night.