WikiTree PhotoTag 50 分鐘內為約 2,500 張人臉完成命名，照片全程留在本機

This is my submission for the Hacktoberfest Open-Source AI Challenge: Week 3, theme "Slow and Steady".

What I Built

My family album is 2,470 scanned photos in about two dozen folders: school portraits, 1940s snapshots, and plenty of "copy 2" duplicates. I also manage family profiles on WikiTree, and every one of those profiles would be better with a photo. But the album is full of people who are still alive, and putting their faces on a public website is not my call to make. So the photos have sat on my disk for years.

WikiTree PhotoTag does the dreaded part:

1. It scans the album and finds every face.
2. It groups look-alike faces, so I name a group once instead of a face at a time.
3. I tell it who each group is, using the people in my WikiTree CC7 export (everyone within seven connections of me).
4. It exports only the photos where every face is named and every person is deceased, one folder per WikiTree profile, with a ready-to-paste caption like [[Id|Name]] with [[Id|Name]], 1955.

Everything else stays on my machine. Uploading to WikiTree stays manual, which is how WikiTree wants it.

Demo

All screenshots are from my real album, taken with the tool's own --blur mode. Every face is pixelated, and living relatives' names are hidden, except for named deceased relatives of an older generation (the tool can keep anyone born after a year you choose hidden too).

Name faces: each card is one group of look-alike faces. I click the right person, or one of the suggestions. The ✎ marks a scan with writing on it.

Named people: everyone named so far, with deceased people marked as exportable. I can select single faces here and move them or send them back.

Find a person: pick a relative and see every unnamed face the model thinks is them, best match first. (These candidates are pixelated because they are not named yet.)

A 19-person family photo. Four older relatives are named. Everyone else is pixelated, including faces the main detector missed: blur mode runs a second, much more sensitive detector and hides anything face-like that is not a named deceased person.

Code

github.com/robinson27225/wikitree-phototag

MIT licensed, about 1,300 lines of Python, 25 tests. Setup is in the README: init, models, scan, cluster, label, export, plus optional read-text and xmp.

How I Built It

The stack is deliberately small: OpenCV for both models, SQLite for the state, a stdlib HTTP server with one HTML page for the UI. No GPU, no framework, no account.

• Faces: YuNet detects them and SFace turns each into a 128-number embedding. Both are small ONNX models from the OpenCV Zoo. The 2,470 photos produced 5,353 faces in well under ten minutes on a laptop. Nine byte-identical duplicate files were skipped automatically.
• Grouping: my first attempt was a greedy "assign to the nearest running average" pass. It turned 3,791 of 5,353 faces into one giant group, because a running average drifts toward a generic face and starts matching everyone. Switching to average-linkage hierarchical clustering (SciPy) fixed it: the largest group is now 326 faces, and they are clearly the same person. Over-splitting is cheap here, because naming a person twice costs one extra click, while a merged pair of different people costs a lot of confusion. I picked the threshold with that in mind. A dev.to post on how a face model change also changes your threshold (by yakhyo) was a useful reminder not to treat 0.36 as a magic number.
• Hints from the tree: the CC7 gives names and dates. Album folders like 0A2 Carl William Robinson Sr and file names like CarlSr_Fran_Teresa 003.jpg become suggestions.
• Narrowing by date: only 8 of my 2,470 scans carry a year in their file name or EXIF, so I build a date window for every photo from several sources. A year in the file name or EXIF is strongest. So is a printed date stamp that Gemma reads off the print ("MAY 64" becomes 1964): that took me from 8 dated photos to 58. Next, the people already named in the photo: nobody is in a picture before they were born or after they died, so one named person bounds who else could be there. Last, a rough decade that Gemma guesses from clothing, hairstyles and photo quality. I tested that guess on the photos that do have a year: it was right for a 1907 and a 1969 photo, but 10–30 years off on several others. So it only adds a loose window (about 30 years either way) and never overrides the tree. Windows only demote candidates ("dates don't fit"), they never hide anyone, because a wrong guess must not hide the right person.
• Gemma reads the writing: Gemma 3 (4B) running in Ollama transcribes handwriting and printed captions, and the tool looks for relatives' full names in the result. I ran it over all 2,470 photos (about 2.5 seconds each, about two hours in total, entirely offline). 257 had readable writing, 184 of them photos with faces, for example "for Grandpa Carl William Robinson / b. 11-13-1933". A surprise: much of the text on scans without faces was documents (certificates, discharge papers, an obituary), not photo backs, because my backs aren't scanned as separate files. The writing on the prints themselves was mostly names, signs, and lab date stamps. A ✎ badge in the UI marks scans that have writing on them.
• Find a person: pick anyone from the CC7 and see every unnamed face the model thinks is them, best match first, then tick the right ones and accept. It is the quickest way to fill out a relative once a few of their faces are named. At a 50% match the top faces were clearly right, but around 40% many were someone else, so I accept by eye rather than by threshold.
• Older groups first: there is no face-age model, so an optional ordering uses black-and-white prints, early dates, the Ancestors/ folder and deceased folder names to put probably-deceased groups at the top. That is where the exportable photos are.
• Set aside: classmates and strangers get one click ("not family"), and for a school photo, one click sets aside every face still unnamed in it. They are never matched again, and any photo containing one stays private, since a stranger could be living.
• Non-destructive, and XMP tags: originals are never touched. Exports are copies with an .xmp sidecar holding standard face regions, the people in the image, a caption and the WikiTree IDs. A separate phototag xmp --write command can tag the album itself with a <photo>.jpg.xmp beside each original (1,503 of my photos have named faces), so the names show up in digiKam and similar tools. It is a dry run by default and never overwrites a sidecar that already exists.

The part that matters: the privacy gate

The CC7 has no "living" flag, so the tool treats someone as deceased only if they have a death date, or were born before 1910. Then the export rule is a few lines:

unknown = sum(1 for f in real if not f["person_id"])
others = sum(1 for f in real if f["person_id"] == OTHER)
living = [pid for pid in d.people if not roster.is_deceased(pid)]
if unknown:
    d.kind, d.reason = "unnamed", f"{unknown} face(s) not yet named"
elif others:
    d.kind, d.reason = "other", f"{others} face(s) set aside as not family (could be living)"
elif living:
    d.kind, d.reason = "living", "possibly living: " + ", ".join(living)
elif not include_auto and any(f["label_src"] == "auto" for f in real):
    d.kind, d.reason = "auto", "contains names nobody confirmed (auto-accepted)"
else:
    d.export = True

A photo leaves only if no face is unnamed, no face is a stranger, nobody is possibly living, and every name was confirmed by a human. Unknown classmates in a school photo keep the whole photo private.

What went wrong (and changed the design)

• Folder names are hints, not facts. A folder named for a relative contained mostly photos of children. A group of them was auto-suggested as that relative. Names guessed by the tool are now marked auto and never exported unless I pass --include-auto. A wrong guess could otherwise file a living child under a deceased person's profile.
• First names lie. Two family members share a name, and the living one is not in my CC7. Weak first-name matches now get a dashed "(weak)" button, and picking a deceased person asks for confirmation. I made exactly this mistake once while labeling (I named a living relative as the deceased one), so the Named people tab now lets me select individual faces and move them, or send them back to unnamed.
• The detector misses faces. The privacy gate only knows about faces that were detected. The README says so, and the demo blur uses the more sensitive second pass for that reason.
• A tidy-up broke the app. I cleared my Downloads folder and the CC7 file the tool pointed at was gone. Now init keeps its own copy, and a missing file gives a clear message instead of a crash. My labels were never at risk, because they live in a separate database.
• My own screenshots nearly leaked. The first blur mode hid faces but still printed living relatives' names on screen, and it showed a relative who died this year. Blur mode now hides living names, and --clear-born-before keeps younger deceased relatives pixelated too.

Why Does Open Innovation Matter?

Because the data is the problem. The photos show real people, many of them alive, and my relatives did not sign up to be uploaded to a cloud vision API. With open models, the whole pipeline (detection, recognition, caption reading, even the era guess) runs on a laptop with the network off.

Open also meant I could change the parts that mattered. I could swap the clustering method when the first attempt collapsed, set my own match threshold, run a second detector purely to over-blur, treat Gemma's era guess as a loose hint after measuring how wrong it can be, and read the CC7 file as plain data. A hosted face-tagging service decides all of those for you, and a "this is a family member" feature would have been the opposite of what I needed: a way to be conservative, not convenient.

There's a community example of the same instinct: Indexatron by swmcc, which uses local models to analyse family photos for the same reason.

The Lazy Payoff

• Without the tool: tag 5,353 faces one at a time, and check each against 4,131 people in the tree.
• With it: 81 group decisions covered about 2,500 faces in about 50 minutes, roughly 31 faces per click. Setting aside strangers (classmates in school photos, for example) is one more click.
• What I got: photos of deceased relatives, sorted into WikiTree-ID folders with captions, ready for upload. At the time of writing, 81 photos covering 15 deceased relatives are exportable, while 767 stay private because they include someone treated as possibly living.
• What I did not do: name the other ~2,750 faces. They are mostly small groups and one-offs, and anything unnamed simply stays private. That is the point of the design: an unfinished job is still a safe one.

Prize Categories

Best Use of Gemma: Gemma 3 (4B) runs locally through Ollama. It reads handwritten and printed captions and date stamps on the scans (257 photos had writing, 50 gave an exact year), and it makes a rough guess at the decade of every photo, which narrows who can appear in it. Those names and dates feed the suggestions. I'm not using it for the face matching itself, which is what the OpenCV models are for.