Doorman：作者六週收到的 23 條聯絡表單訊息全是推銷

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

I have three small Shopify apps. Their listings have a contact form, and between 29 August and 13 October that form and the email address behind it got 35 messages: 23 different ones once the copy-pasted repeats are gone, from 21 senders. All were selling something. Installs and reviews "from real store owners" at $30 a store. A paid post in a Shopify Discord. UGC videos. One opened with "are you the right person in charge of this brand...?", which sounded like it might be a question, so I answered it. It was a pitch.

The problem isn't the pitches. It's the day a real merchant writes "the cleaner deleted a product I wanted to keep, can I get it back?", and it sits between two messages that begin "I came across your Shopify app and was really impressed".

Doorman sorts a contact-form inbox: customers first, pitches folded away. Drop in .eml files or a Gmail Takeout .mbox, or paste a message, and press Sort. Every pitch is folded away with the sentence that gave it away highlighted; close calls stay on top, marked "not sure"; nothing is ever deleted or answered. It runs entirely in your browser. A customer's message has their name, their store and their order in it, and none of that leaves the page.

Who would pay for it: anyone whose contact form is public and whose customers are worth answering fast. Indie app developers, Etsy and Shopify sellers, the person who runs a small SaaS from a support@ address. The pitches find them all, because a listing is a public lead list. I'd pay a few dollars a month for this to run on my own inbox by itself, and that's the version I'd sell.

Demo

Live: doorman.mikey9220.workers.dev. Press Load Example Inbox, then Sort Inbox. The example is five merchants' questions from the public Shopify forum and seven pitches from my own inbox (anonymised). The first sort downloads EmbeddingGemma once (197 MB); after that, on my Mac in Chrome, the twelve messages take 2.1 seconds on the GPU.

Code

mikey92 / doorman

Sorts the contact-form inbox of a small app or store: real customers in, sales pitches out. Gemma runs in your browser; messages never leave your machine.

Doorman

Your app's or shop's contact form, sorted: customers first, sales pitches folded away. It runs in your browser; nothing you load leaves the page.

Live: https://doorman.mikey9220.workers.dev · Evaluation: eval/CLASSIFIER.md · Data: eval/DATA.md

In six weeks my three Shopify apps' contact form, and the email behind it, got 23 different messages from 21 senders. All 23 were people selling me something: installs, reviews, a promo post in their Discord. Not one was a customer. The day one is, I don't want it buried under the next "I came across your app and was really impressed".

Load messages (paste one, drop .eml files, or a Gmail Takeout .mbox) and press Sort Inbox. EmbeddingGemma 300M, running in a Web Worker on your GPU, turns each message and each of its sentences into 768 numbers; a logistic regression trained on 245 hand-labelled forum posts and my inbox's pitches says how…

How I Built It

.eml / .mbox / pasted text → the text part, minus the contact form's wrapper and quoted replies
→ EmbeddingGemma 300M in a Web Worker on WebGPU: the message, and each sentence, as 768 numbers
→ a logistic regression (768 weights and a bias, 7 KB of JSON): P(pitch)
→ P ≥ 0.8: folded away · 0.5–0.8: "not sure", kept on top · below 0.5: customer
→ the most pitch-like sentence is quoted on the card

A static site on Cloudflare's free tier: Transformers.js runs onnx-community/embeddinggemma-300m-ONNX (q4) with the prefix EmbeddingGemma uses for classification, task: classification | query:. No server.

That isn't where I started.

First try: ask Gemma 3 1B, and read its mind

My first version was the obvious one. Last week's project ran Gemma 3 1B in the browser, so I asked it, for each message, to copy the sentence where the sender offers their own product or service, or write NONE.

Gemma 1B doesn't copy. Asked for "word for word", it paraphrases: "OFFER: StartStorez Bundle is a free bundle app" for a message that said "you can give StartStorez Bundle a try". Every quote failed the check.

So I stopped asking it to write anything. For each sentence I asked "In this sentence, is the sender offering the business their OWN product, app, service, agency or help? Answer Yes or No", and instead of reading its answer, read its next-token logits for "Yes" and "No". With an open model you can do that: the generate loop takes a logits processor, and mine only looks.

const peek = (_ids, logits) => {
  pair ??= [logits.data[yes], logits.data[no]];
  return logits;
};
await model.generate({ ...inputs, max_new_tokens: 1, do_sample: false, logits_processor: [peek] });
return pair[0] - pair[1]; // log-odds of Yes over No

The logits told me something the answers never would have: Gemma 1B says Yes to almost everything. On 40 hand-labelled messages, customers scored between −2.3 and 11.6 in log-odds; pitches between 6.8 and 14.9. A merchant writing "I really want to maximise our AOV with the busiest time of year approaching" scored 6.9, a 99.9% Yes. It ranks pitches above customers more often than not (AUC 0.91), but only 9 of 20 pitches scored above every customer, and with one forward pass per sentence it took about 20 seconds a message on my Mac's CPU.

What worked: don't ask, measure

A pitch doesn't need to be understood to be recognised. "I came across your app", "I can help you get your first installs", "let me know if you're open to a quick chat": pitches read alike, and an embedding model is built to notice what reads alike. EmbeddingGemma is a quarter of the download, and one forward pass gives me a message's 768 numbers. All I needed was examples to fit a line through them.

Where the examples came from

My inbox had 23 pitches and no customers, which is the problem but isn't a dataset. So I went where merchants ask for help in public: the Shopify Community "Shopify Apps" board. I took the 60 most recently active topics, 637 posts, and read and labelled every one: asking for help, selling something of their own, helping with nothing to sell, or other.

282 of the 637 posts were someone selling their own app or service. 48 of the 60 topics had at least one. One merchant asked for bundle app recommendations "(from shop owners, not app developers!)"; seven of the thirteen replies were from app developers.

I kept the confident calls (150 "is this account really the app's maker?" calls went out), one pitch per account so a vendor who posts the same template eleven times doesn't grade the classifier on that template, and got 115 merchants asking and 130 pitches. (How it was labelled.)

How well it works

Five-fold cross-validation over the forum posts, split by topic so nothing is ever scored by a model that saw its thread (full report):

AUC 0.991. On the same 40 messages Gemma 1B judged, it's 1.000: the most pitch-like customer scores 0.28 and every pitch is above 0.8.

The page's bar is 0.8 because hiding a customer costs a sale and leaving a pitch on top costs five seconds. The 23 pitches between 0.5 and 0.8 stay on top as "not sure", with the sentence that looked most like a pitch quoted.

The two customers it would hide are worth reading. One is a refill shop whose forum signature says "Award-winning refill shop in Oxfordshire. Wide range of refillable, eco-friendly, vegan & organic products." The other, answering a thread about bundles, describes their own product line in the same voice. A merchant describing their own shop sounds like a pitch, because it is one, just not aimed at me. That's the case "not sure" is for, and it's why nothing is ever deleted.

My inbox: a lesson in where training data comes from

Then the test that mattered. Trained on the forum alone, the classifier folded away only 13 of my 23 pitches at 0.8 (22 at 0.5). Cold emails are written differently from forum replies: longer, more flattering, and they end by asking for a call. The fix was to train on them too, and to test it honestly I scored each of the 21 senders with a model trained on the forum and on every other sender's messages. 23 of 23 folded away at 0.8, 19 at 0.9. The shipped classifier is trained on all 268 messages.

Why Does Open Innovation Matter?

The messages are the private part. A support inbox is customers' names, emails, store URLs and order problems. With an open model in the browser, sorting it means nothing is uploaded, and there's no server holding anything. "Your customers' messages never leave your machine" is the whole pitch of a product like this, and with a hosted API I couldn't say it.

It's free to run, so it can be cheap to buy. A message takes under a fifth of a second of the visitor's GPU. A hosted API would put a bill on every message, a key behind a server, and a floor under the price.

I could look inside. Reading Gemma 1B's logits is what showed me it was saying Yes to everything, a thing a chat API's text answer would have hidden behind a confident "Yes". And open weights let me train my own 7 KB head on EmbeddingGemma's vectors, ship it as a JSON file anyone can read, and run the exact same code in Node for the evaluation.

Where a closed model would be better: a frontier model would likely sort these well without a single labelled example. But it would see every customer's message, charge for every one, and I'd still have needed the 245 labels to know how well it does.

My First User

Me. Doorman's classifier sorted my own inbox: all 23 pitches folded away, each sender scored by a model that had never seen them. No customer had written yet, so there was nothing to keep on top.

What I'd build next, and what I'd charge for: a Chrome extension that runs the same model on the Gmail page you already have open, so the pitches fold away without exporting anything. Same model, same 7 KB of weights, still nothing uploaded.

Prize Categories

• Best Use of Gemma: EmbeddingGemma 300M does all the reading on the visitor's GPU; Gemma 3 1B's logits showed why asking a small model "is this a pitch?" fails.