跳到正文
DEV Community· Carlosjv·· 1 天前AI 評分38

Stillroom:檢視重複通知,同時保留原始訊息

Stillroom: review the repetition without losing the message

AI 導讀

Stillroom 將貼上的英文通知整理成可供人工檢視的分組,同時保留原始訊息,並容許用戶拆開錯誤分組。在 36 條合成訊息測試中,加入保守規則後,分組精確率由 22.2% 升至 75.0%,召回率則由 100% 降至 75.0%。工具在瀏覽器本機推理,測試結果支持它作為檢視輔助,而非自動去重工具。

正文

Built for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady.

What I Built

Take two synthetic notifications:

“Your appointment is confirmed.”

“Your appointment is cancelled.”

An embedding model can consider them very similar. For someone reviewing notifications, the difference is the point.

Stillroom turns pasted English notifications into a review queue. It proposes groups, keeps every original available, and lets the reader split a mistaken group. Headings and excerpts come from source text, with message identifiers. There is no generated account of what supposedly happened.

The intended benefit is less manual sorting before a review session. The first evaluation showed why a shorter queue was not enough, and that finding changed the design.

Demo

Open Stillroom

Choose Load sample, then Find the common threads. The sample is fictional; the demo uses the guarded method reported below. Inspect the original messages in each group and choose Keep these messages separate if a grouping is unhelpful. You can compare with word matching, mark cards as reviewed, and export the review as JSON.

Paste lines, JSON or CSV: up to 60 messages, with 4,000 characters per message. Import trims leading and trailing whitespace. Stillroom has no inbox connection and performs no send, archive or delete actions.

Code

Source and reproduction instructions · Download source ZIP · Evaluation evidence

Application development began on October 21, 2026. The new application code is MIT-licensed. Pretrained weights and runtime libraries retain their own licenses and attribution. The ZIP includes the application, tests and evaluation material; a download script retrieves the pinned model/runtime assets and checks their hashes.

How I Built It

The semantic component is all-MiniLM-L6-v2, an Apache-2.0 sentence embedding model, using the Xenova ONNX conversion. It turns short messages into 384-dimensional vectors, allowing differently worded messages to be compared by meaning.

The browser runs quantized inference through Transformers.js and ONNX Runtime WebAssembly. The weights are about 23 MB; tokenizer and runtime files add to the download. The encoder uses at most 256 tokens per message, while the full imported text remains available. This prototype targets short English notifications.

The grouping rule is complete-link clustering: every pair in a group must meet the similarity threshold. That prevents a loosely related message from bridging otherwise dissimilar groups. Fixed evaluation thresholds are 0.68 for MiniLM and 0.32 for a TF-IDF word-overlap baseline. Neither is a probability.

The first test contained 48 synthetic messages in two scenarios. The unguarded model found all 30 labelled same-family pairs, but also grouped 47 pairs from different families: 39.0% pairwise precision. It produced only 15 cards. That compactness came with too many incorrect merges.

The revised method adds conservative rules for explicit numbers, recognized currencies, English date terms and identifiers in the text. It also checks three opposite-status pairs and the presence of explicit negation. A pair must pass these rules as well as the unchanged similarity threshold.

The guard can separate compatible messages when a detail is absent or expressed differently. It can also miss differences, including values assigned to different entities. It does not determine truth, chronology or which update is current.

A fresh test used 36 synthetic messages, with eight labelled same-family pairs. The corpus and labels were frozen before the evaluator read the implementation; the core was frozen before inference. A family means the same event and substantive payload, not merely a shared topic. All three methods received the identical new set:

Measure Word matching MiniLM alone MiniLM + guard
Correct pairs grouped 8 8 6
Incorrect pairs grouped 28 28 2
Same-family pairs missed 0 0 2
Pairwise precision 22.2% 22.2% 75.0%
Pairwise recall 100% 100% 75.0%
Review cards 16 16 29
Messages retained once 36/36 36/36 36/36

The guard traded compression and recall for fewer incorrect merges on this set. It still mixed successful and declined rent-payment notices. It also separated equivalent 24-hour/12-hour times and two cancellation notices phrased differently. It failed the test’s zero-false-merge target. The result supports a review aid, not automatic deduplication.

The two unguarded methods returned identical counts; this test does not establish MiniLM’s superiority over word matching. Nor should its results be compared numerically with the earlier 48-message test as if the datasets were interchangeable.

The evaluation ran the shipped JavaScript/WASM worker in a Node VM with local Web API adapters. It checked source coverage through the core and digest; it was not a browser usability test. Labels were authored by an AI agent, without independent human validation. No thresholds or rules were changed after these results.

Why Does Open Innovation Matter?

Local inference lets Stillroom process notification text in the browser without sending it to a hosted language-model API. No secret API key or per-message inference purchase is needed. Initial application and model downloads require connectivity; fully offline operation is not claimed.

Open weights and source make the mechanism replaceable and the failures inspectable. Someone can reproduce the experiment, change the encoder, or challenge the grouping rules. That transparency mattered here: testing exposed weaknesses that a calm interface could easily obscure.

The human still decides what each message means. Fewer cards are not measured time saved, and preserved sources are not a guarantee that every important change will be noticed.

Prize Categories

Overall prize only. No partner category is claimed.

Build disclosure: Codex assisted with the concept, implementation, evaluation and article. AI assistants are development tools, not additional human team members. No personal field testing or measured productivity benefit is claimed.

來源:DEV Community · dev.to