Second Look 推出，以 758 人實際醫療支出評估 2027 醫療計劃選擇

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

When I came to the US for grad school, I signed up for a health plan just to have one. I didn't know what a deductible was, or a copay, or coinsurance. I wasn't choosing a plan. I was ticking the box that said I was covered.

A friend of mine picked their plan at work a different way: they chose the one everyone on their team had chosen.

That's most people. In a study of 23,894 employees at one large US company, most chose a plan that another option on the same menu beat no matter what happened to their health, and they overpaid by about 24% of their premiums. Most of them didn't switch the next year either (Bhargava, Loewenstein and Sydnor, QJE 2017). And 2027 is not a cheap year to coast: employers expect health costs to rise 8.2%, the biggest jump since 2003 (Mercer).

So I built Second Look. It checks your plan choice against what really happened to real people. You pick your employer's plans (or drop in the plan PDFs) and answer five questions. Second Look tells you whether the lazy choice is safe for you.

Along the way it lost an argument I was sure I'd win. That's the most useful part.

TL;DR. I priced the real 2024 medical care of 758 people (from a US government survey) under three employers' real 2027 plan menus.

• "Just take the lowest premium" got the truly cheapest plan for 82% of them, and overpaid $106 on average.
• "Same as last year" got it for 78%, and overpaid $149. One in ten overpaid $608 or more.
• My TabPFN model landed in between, at 79% and $121.

So Second Look doesn't try to out-think the lazy rule. It tells you when the lazy rule is right for someone like you, and when your costs are too hard to predict to trust it.

Built with PriorLabs-TabPFN. Gemma 4 reads the plan PDFs on your laptop, and code checks every number against the document. TabPFN predicts a range for your next year. Plain code does the dollars. Everything runs locally.

▶ Watch the 21-second launch video, made from the real app and the real numbers.

Pick a real 2027 menu, pick a real (anonymous) person, get a verdict.

What I Built

Second Look answers one question: is the lazy choice safe for me this year?

1. Pick your employer's 2027 plans. The demo has three real menus: Washington State (PEBB, 9 plans), Oregon (PEBB) and the State of Tennessee. In local mode you can drop in any plan's Summary of Benefits and Coverage, the federal form every US plan has to publish.
2. Answer five questions, or pick one of three real (anonymous) people from the survey.
3. Get a verdict:
   • "You can be lazy: plan is right for people like you." Then what each plan costs in a typical year and in a bad year (1 in 10).
   • Or "Look closer this year: your costs are hard to predict." You get the same comparison, but no promise of a saving the numbers can't back.
4. Tap any plan number and you see the line in the plan PDF it came from, highlighted, with the page.

A real (anonymous) healthy adult on Washington's 2027 menu: about $80 in a typical year on the lowest-premium plan, against $1,120 on UMP Select (and $1,880 on UMP Classic, further down). Each bar is 12 months of premiums plus their share of care, less the employer's HSA deposit. The solid bar is a typical year; the outlined bar is a bad year (1 in 10).

Who gets to be lazier: anyone on an employer plan who has ever clicked "keep my current plan". Five answers instead of reading nine eight-page PDFs. For most people the answer really is "stay lazy, take the cheap one". Now you know it's true for you, and what a bad year would cost.

Demo

Live: hotragn.github.io/secondlook (static, nothing to install, real 2027 plan documents)

Tap a deductible and you get the exact sentence from that plan's 2027 PDF, highlighted, with the page.

A few things to try:

• Pick Washington, then the "ongoing prescriptions" person. The verdict flips to "look closer".
• Tap any deductible or copay. You get the exact sentence from that plan's 2027 PDF, highlighted.
• Open "What actually happened in 2024". It shows each person's real care, priced under every 2027 plan.
• Open "How sure is this?" It has every number below, with its limits.

Code

Hotragn / secondlook

Check your 2027 health plan against what really happened. Local Gemma 4 reads the plan PDFs, TabPFN predicts next year, code does the dollars.

Second Look

Check your 2027 plan against what really happened.

Second Look tells you whether the cheapest 2027 health plan on your employer's menu is safe for you.

• When it is, Second Look says so, and shows a typical year and a bad year (1 in 10) for every plan.
• When your costs are hard to predict, it tells you to look closer instead of pretending to know the winner.
• Every plan number links to the line in the plan document it came from, highlighted, with the page number.

Second Look checks your plan choice against what really happened to 758 real people in 2024.

Built with PriorLabs-TabPFN. Plan documents are read by Gemma 4 running on your own computer.

Results

Each evaluation was frozen before scoring, and each test set was scored once. Every number below links to the JSON file it comes from. eval/NUMBERS.md gives the exact key…

How I Built It

plan PDFs ─▶ pdfplumber text ─▶ Gemma 4 E4B (Ollama, JSON schema): {quote, page} per field
          ─▶ code: is the quote in the PDF? read the number out of the quote
          ─▶ the PDF's own worked examples (Peg, Joe, Mia) checked against what was read
five answers ─▶ TabPFN (3,033 real people, 2023 → 2024) ─▶ a range for next year
          ─▶ each plan's rules (deductible, copays, coinsurance, out-of-pocket cap, HSA)
          ─▶ typical year and bad year per plan ─▶ verdict

The data is all public:

• MEPS, the Medical Expenditure Panel Survey from the US Agency for Healthcare Research and Quality. Panel 28 follows the same people through 2023 and 2024, and the 2024 visit-level files price every office visit, prescription, ER visit and hospital stay.
• 45 real plan PDFs from 11 public employers, including 20 for calendar 2027.
• Their published premium sheets.

The plan PDF grades the reader

Every Summary of Benefits and Coverage uses the same federal template, so Gemma has a fixed list of fields to find. The rule I held it to: it may only point at text.

• For every field it returns a quote and a page.
• Code checks the quote is really in the PDF, and reads the number out of the quote itself.
• Gemma never types a dollar amount. No quote means the field shows "check this", never a guess.

The form has a bonus: every plan prints what three made-up members would pay. Peg is having a baby, Joe has type 2 diabetes, Mia breaks a bone. Second Look checks what it read against those printed totals, so the document grades the reader. When a value was wrong, those checks flagged the document most of the time. For E4B, that was 6 of the 8 documents with an error.

I froze the 20 calendar-2027 PDFs as a test set before scoring anything. The answer key, 260 values, was committed before any model saw them. Then I scored each model once:

Three Gemma 4 sizes on the 20 frozen 2027 plans, each scored once. Source: eval/results/step3_reader.json

The 26B model bought nothing over E4B, so E4B ships.

The real misses:

• Tennessee's tables put a $2,400 specialty-drug out-of-pocket limit next to the medical one, and the models grabbed the wrong one.
• One Tennessee PDF says "$1.800", a typo for $1,800, and it was read as $1.80.
• "15% coinsurance after $75 copay" broke my parser.

In the app, those values show the correct line with a "check this" note instead of pretending.

The Gemma gotcha that cost me an hour: on Apple Silicon, the default gemma4:e2b and gemma4:e4b tags pull MLX builds that refuse a JSON schema ("structured output is unavailable"). The QAT builds (gemma4:e4b-it-qat) run on llama.cpp and honour it. The whole reader depends on that.

TabPFN predicts next year, not last year

TabPFN learns from a table you hand it at prediction time, with no training loop. I gave it 2,275 real people's 2023 facts (age, health, visits, prescriptions, spending) and asked for their 2024 spending as a range. Then I scored it on 758 people it had never seen, with households kept together so family members couldn't leak between the two sets.

• Against "next year = last year", TabPFN's middle guess was off by $1,185 less per person (95% CI $734–1,642).
• Against the average for your age and sex, it was $461 less.
• Against gradient boosting and linear quantile regression, it tied.
• Its 80% range caught 77% of real outcomes, slightly narrow.

The learning curve is where it earned its place:

• It reached its full-data accuracy with 200 people. Linear regression needed all 2,275.
• With 400–800 people, its bad-year (90th percentile) guesses beat gradient boosting by $215–235.
• Its ranges stayed calibrated (74–80% coverage) at every size, where gradient boosting's sat at 63–73%.
• At 100 people, gradient boosting was better. I'm keeping that line in.

Same frozen test set, training on 100 to 2,275 people, 5 random draws per size. Source: eval/results/step2b_learning_curve.json

Code does the dollars

Deductibles (including family deductibles that apply per person), copay or coinsurance, before or after the deductible, the out-of-pocket cap, $0 preventive care, the employer's HSA deposit and the tax you save on your own HSA money. The 2027 limits are pinned to their sources:

• HSA: $4,500 single / $9,000 family.
• HDHP minimum deductible: $1,750 / $3,500 (IRS Rev. Proc. 2026-24).
• The ACA out-of-pocket ceiling: $12,000 / $24,000.

There are 41 tests. Three of them reproduce a real plan's printed Peg, Joe and Mia examples to the dollar.

The argument I lost

This was supposed to be the headline: "TabPFN picks a better plan than you do." So I priced every test person's real 2024 care under every plan on three real 2027 menus, added a year of premiums and subtracted the employer's HSA deposit. Then I asked each picking rule how often it found the truly cheapest plan.

758 held-out people × 3 real 2027 menus, priced on their actual 2024 care. Source: eval/results/step4_decision.json

The laziest rule won. On these menus, a high-deductible plan with an employer HSA deposit was the cheapest plan for most people (576 of 758 in Washington). "Same as last year", the rule almost everyone actually uses, was the worst sensible one.

I tried once more, with the test set still sealed:

• I trained a TabPFN classifier to spot the people the lazy rule costs more than $250. That's 11% of cases.
• It was the best of five methods at that (AUC 0.76, against 0.75 for logistic regression and gradient boosting).
• But swapping in TabPFN's pick for the people it flagged made things worse: $119 against $106.
• Even with perfect flags, the gain would only be $9.

The honest product is: take the lazy rule when it's safe, and when it isn't, show the ranges instead of a made-up saving.

Bugs I caught before they shipped

• A misaligned comparison: a paired-difference calculation in the TabPFN benchmark was lining up the wrong rows. It was caught before commit, and the fix is in the history.
• A stuck local server: a week-old orphaned Ollama process loaded no models at all ("failed to create library"). A fresh server loaded the same model in 2.3 s.

Why Does Open Innovation Matter?

• Your health and your income stay on your laptop. Five answers about your health and your money are exactly what you shouldn't paste into a chatbot, and many employers don't allow it. Gemma 4 runs through Ollama and TabPFN runs on the laptop's own chip. The demo site makes no network calls at all.
• You can check every number. The models are open, the survey is public, the plan PDFs are public, and every value links to its sentence. When Second Look is wrong you can see where, like the Tennessee typo above.
• It costs nothing to come back next year. Open enrollment comes back every fall. A tool you run yourself doesn't need a subscription for one evening a year.

The Lazy Payoff

A persona, not a real user (yet)

I don't have a friend choosing from these exact menus this week, so I made one person up. She's built from my friend's habit and my own first year in the US. She is not real. Her numbers are exactly what the live demo shows for her five answers.

Meera, 28, works for a Washington state agency in Vancouver, WA:

• healthy, no ongoing conditions, a few prescriptions a year, no ER visits
• last year she picked UMP Classic, because most of her team was on it
• this fall she was about to click "keep my current plan"

Second Look tells her: "You can be lazy: Kaiser Permanente NW CDHP (the lowest premium) is right for people like you."

• A typical year: about $1,030 on the CDHP against about $2,140 on UMP Classic, so roughly $1,120 less.
• A bad year (1 in 10): about $2,540 against $3,160.

How I imagine it landing for her (my words, not a quote): I thought the cheap plan was the risky one. Seeing the bad year next to the normal year, the expensive plan was mostly buying peace of mind I wasn't using.

Three real people from the survey

The other "friends" are three real people from the survey's test set, and their 2024 care actually happened:

• A healthy single adult on Washington's menu: the lazy pick (Kaiser NW CDHP) costs about $80 in a typical year, against about $1,880 on UMP Classic. Priced on what really happened to them in 2024, it would have cost −$60 (the HSA deposit covered more than their care), against $1,780 on UMP Classic.
• Someone with ongoing prescriptions, and a high spender: flagged in Washington and Tennessee. Second Look shows the ranges and stays out of the guessing.

On the live demo, five answers take about a minute. The other way is reading nine plan PDFs, eight pages each, and doing the maths yourself.

My Agent Session

I built Second Look with two Claude Code sessions: one as the manager, holding the research and the decisions, and one as the builder, getting one short prompt at a time. Three moments mattered most:

1. Freezing the 2027 plans as the test set before writing the reader, so the reader was scored on employers and documents it had never seen.
2. The builder refusing to retune on the test set after the lazy rule won. It reported the loss and asked what the product should do instead.
3. Pre-registering the "lazy-safe" thresholds ($250 and $500) in writing before scoring them.

What It Can't Do (yet)

• Self-only coverage. Family plans are next.
• It doesn't know if your doctor is in network.
• It models 88% of 2024 spending: dental is excluded, since it isn't part of a medical plan, and some smaller categories aren't covered yet.
• A typical year leans on the five most similar survey people, so one plan can move by a few hundred dollars.
• The reader's answer key (260 values from the 20 PDFs) was transcribed by my coding agent, not checked line by line by a person. 25 values are marked ambiguous, and the method is in eval/README.md.
• It shows costs. It doesn't tell you what to buy.

Prize Categories

• Best Use of TabPFN:
  • It predicts each person's next-year spending as a range, from 2,275 real people.
  • It's benchmarked against four simpler predictors and a learning curve: full accuracy from 200 people, and calibrated ranges at every size.
  • Its classifier powers the "look closer" flag (AUC 0.76, the best of five).
  • Its loss to the lazy rule is reported in full.
• Best Use of Gemma:
  • Gemma 4 E4B reads the federal plan form locally and may only quote.
  • It's checked against the plan's own worked examples.
  • Three Gemma 4 sizes were scored once on 20 frozen 2027 plans: 91.9% correct (98.7% excluding ambiguous fields).
  • Plus the MLX-versus-QAT structured-output gotcha.

Built with PriorLabs-TabPFN. Built with help from Claude Code. Data: MEPS (AHRQ) public-use files and public employer plan documents. Models: Gemma 4 QAT builds (Google, Apache 2.0) and TabPFN v2 (Prior Labs License 1.1).