LazyShelf:尋找段落,保留來源
LazyShelf: find the passage, keep the source
LazyShelf 是一款搜尋英文 TXT 和 Markdown 文件段落的工具,讓用戶查看結果的原文來源及上下文。它提供 BM25 式 Words 排序和 Meaning 語義搜尋,後者使用 Xenova/all-MiniLM-L6-v2,透過 Transformers.js 3.8.1 執行。
This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady.
What I Built
You know the note exists. You remember the idea, but not the words you used.
LazyShelf is a small passage finder for a collection of English text and Markdown files. Add your documents, search for a word or describe what you remember, and open a result in its original context. It is designed for people with folders of notes, instructions and saved text who would rather find a passage than reopen every file.
For this week's Slow and Steady theme, the work to avoid is reopening and rereading files one by one. LazyShelf retrieves a candidate; you keep the judgment by checking its separate source identity and exact original context.
It does not write an answer. It returns text that was already in your documents, with the filename, a collection-specific file ID and a source line. Two files named shift-notes.txt remain two different sources.
The deliberately narrow scope is the point: UTF-8 TXT and Markdown, a small collection, and a source you can inspect. There is no document chatbot, PDF parser or account system.
Demo
The demo is a static GitHub Pages deployment. Start with Try a fictional collection; Words mode works without loading the model. Meaning mode downloads the runtime/model assets from the same site and performs inference in your browser. First preparation is not instantaneous. Imported documents and queries are not sent to a hosted inference API; the hosting provider still receives ordinary site/asset requests. You can also run the source locally as described below.
To try the main workflow, add a small collection, describe a passage in Meaning mode, then open its exact source. Switch to Words when you have a distinctive identifier. Read the surrounding context before treating a result as evidence for an answer.
Code
Source repository and installation instructions
Reviewed deployment source: 69fb601a8cda9a70a5e371a75bce5cf0b187e8bb. Its extracted source archive passed all 32 automated tests. The earlier 29-test release remains in Git history; the deployment update makes asset paths work at both the local root and the GitHub Pages project path. The repository includes the fixture and evaluation output discussed below.
The application code uses the MIT licence. Transformers.js uses Apache-2.0; ONNX Runtime uses MIT. The model card at the pinned Xenova revision identifies the model licence as Apache-2.0. The source retains the full runtime licence texts, model revision and upstream ONNX third-party notices. Those obligations are separate from the application licence.
How I Built It
LazyShelf has two retrieval paths:
- Words: a local BM25-style token ranking. It is useful when you know a distinctive term or identifier, and it does not need the embedding model. This is token matching, not a strict phrase-search engine.
-
Meaning:
Xenova/all-MiniLM-L6-v2, a compact open-weight embedding model, running through Transformers.js 3.8.1. It represents a query and document passages as vectors and ranks their similarity.
The browser worker performs the embedding work away from the main interface. The application preserves original decoded text and records each passage's source offsets. Opening a result shows the full document with that exact slice highlighted. It renders imported text as text, including Markdown and HTML-looking strings, rather than executing it.
The collection stays in page memory. Resetting or reloading discards it; LazyShelf is not a backup system. For local installation, a separate setup command downloads about 47.6 MB of runtime and model assets, checking each against a size-and-SHA-256 manifest before installation. The browser worker loads those assets from the same site or local server rather than calling a hosted inference API. Keyword search does not need the model assets.
The source is divided into a text/retrieval core, the embedding worker, UI state management and a small loopback static server. Collection changes invalidate the index and old results. Cancellation retires outstanding work instead of allowing an earlier search to overwrite the new collection. Source-line tracking uses a forward-only cursor so it does not repeatedly scan the full prefix for each overlapping passage.
Try the small, useful version
Download the source repository and extract it, or clone it. From the project directory, with Node.js 22 or newer:
node scripts/download-assets.mjs
node scripts/verify-assets.mjs
node scripts/serve.mjs
Open http://127.0.0.1:4187/ in a modern browser with WebAssembly support. No API key, GPU or npm installation is required for this path. The download step needs internet access; it is not part of every search.
Click Try a fictional collection, then Load semantic model. Once ready, choose Meaning · semantic and search make a backup of my photographs. Inspect the Photos.md passage about a second copy on a separate external drive, then click View exact source. For an identifier rather than a paraphrase, choose Words · lexical and search DVC-0179.
What the evaluation actually showed
A separate reviewer-authored collection contains six original, fictional workshop documents. It includes duplicate filenames, CRLF line endings, Unicode, similar identifiers, negation and literal hostile markup. It contains no patient or customer information.
The fixed evaluation has twenty queries. Seventeen have a specified answer-bearing passage; three ask for information that is absent. The table below uses the standalone Node CPU evaluation of the retrieval core, with the same quantized model. It is not a browser-speed or usability benchmark.
| Retrieval mode | Expected passage ranked first | Expected passage in top three |
|---|---|---|
| Word ranking | 13 / 17 | 15 / 17 |
| Semantic ranking | 16 / 17 | 17 / 17 |
A hit requires the decisive expected text to appear inside a returned passage from the correct source. Merely finding the right file does not count. The semantic top-one miss is retained: the right document appears first, but the complete decisive sentence appears in another passage within the top three.
The three absent-answer queries are not counted as successes. A unique absent literal yields no word matches, but both retrieval methods can return neighbors for a natural-language question whose answer is missing. Meaning search shows the highest-ranked passage with an explicit reminder that it is not a verified answer. Lower-ranked results are collapsed under Other suggestions, with a warning that they may be unrelated. This separates ranking from relevance; it does not turn similarity into an answer-confidence estimate.
The source-only evaluation used thirteen passages and 384-dimensional embeddings, checked every returned source slice, and made zero network requests. Its fixtures, method and actual output are available with the evaluation materials.
These are small, authored tests, not blind user validation or a promise of general accuracy. They do not establish superiority over another application.
The deployment source passed 32 automated tests covering decoding, limits, source coordinates, route containment, model-worker recovery, result presentation, packaging safeguards and root/project asset paths. Its exact Git archive was independently extracted and tested with no skipped tests. The generated deployment files were checked against their manifest. Simulated DOM, loader and worker tests are not browser or accessibility certification; HTTP delivery checks do not establish live browser inference behavior.
What is different, and what is not
SemanticFinder already offers browser-local semantic and full-text search, with a broader collection of features. LazyShelf does not claim to invent either approach.
Its focus is a smaller workflow: import several TXT or Markdown files directly, retain their separate identities, and move from a ranked passage to its exact original context. Word search is available before model preparation and remains the fallback when the model is unavailable. The intended improvement is less preparation and clearer source evidence for this specific task, not an unsupported claim that it is universally faster or more accurate.
The sample check that changed the interface
When I tried the fictional collection with a photograph-backup query, Photos.md appeared first—but the library and equipment passages appeared below it as well. Ranking the right passage first was not enough: presenting every suggestion the same way made the unrelated material look more useful than it was.
We changed the presentation rather than inventing a confidence cutoff. The highest-ranked passage stays visible, while the other suggestions are collapsed and explicitly labelled as potentially unrelated. The original source remains one click away. This was a founder's sample check, not an independent user study, but it exposed a concrete problem and led to a tested fix.
The Lazy Payoff
The intended saving is small and repeatable: stop opening files one by one when you remember the idea but not its wording. Let local retrieval produce a candidate, then spend your attention checking the actual passage. I have not measured minutes saved; the demonstration shows the workflow, not a productivity claim.
Why Does Open Innovation Matter?
Downloadable model weights make the meaning-based path possible without sending each document and query to a hosted model service. Transformers.js supplies the browser inference runtime; the MiniLM conversion supplies the quantized weights; ONNX Runtime supplies execution underneath it.
That separation is useful. Someone can inspect the retrieval code, verify the installed asset hashes, keep the model assets on their own machine and choose word search when meaning-based retrieval is not helpful. A proprietary API key is not part of the application.
Open components do not remove the costs of model preparation, hardware or review. They make those dependencies visible and let this small tool do its job without a per-query hosted inference dependency.
Limits worth knowing
- The quality scope is English TXT and Markdown, not multilingual search, OCR or PDF extraction.
- The collection is bounded: thirty files, 512,000 bytes per file, 500,000 UTF-16 code units overall and at most 900 passages.
- A first semantic index has an upfront cost. Warm searches are a different operation; no universal instant-search claim is made.
- Similarity is not answer confidence. Check identifiers, exceptions and negation in the source.
- A running local server and installed assets are different from an offline-installable PWA. Network-free core evaluation does not prove offline browser navigation or every browser's compatibility.
- The examples and evaluation collection are fictional. There is no measured customer time-saving or production-user claim.
AI disclosure: Codex and Hermes were used extensively for implementation, testing, review and drafting this article. Redwanur Rahman set the product direction and scope. Reported evaluation figures come from preserved execution output, not generated estimates. AI assistance does not transfer responsibility for the entry or its claims away from the entrant.
來源:DEV Community · dev.to