checkwith.us：由家人把關付款的防詐保護

This is a submission for the MLH x DEV Writing Challenge

Every scam script has the same line in it: "Don't tell your family."

So we built the opposite. checkwith.us is scam protection where the older adult does nothing new, and the family does the work. Before money moves, it checks with us.

We built it in 36 hours at SwampHacks XII, as a team of four. The code is public: github.com/builtbyd3v/checkwithus.

What I Built

Most scam tools ask the person at risk to make a decision under pressure. Scammers are very good at pressure. We moved the decision to the people who already worry about them.

The older adult keeps using their iPhone the same way:

• Texts. Our own scam model runs inside an iOS Message Filter extension, on the phone. Scam texts go to Spam, not the inbox.
• Calls. A Call Directory extension labels known scam numbers as "Possible scam".
• Payments. Every payment goes through fixed rules. A risky one pauses, and the phone says: "Your money is safe. Your family will call you."

The family gets the other half: approve or deny each held payment, see every reason in plain words, and freeze the account with one tap.

We set three hard rules before writing any code:

1. AI suggests, code decides money.
2. Money is integer cents everywhere.
3. The older adult's texts and bills never go to a cloud AI model.

The model had to fit where the texts are

The Message Filter is the only place on iOS that sees a text from an unknown sender. So the scam model had to run there, inside a very small memory budget.

Our first model was a fine-tuned DistilBERT. It scored well. At 67 MB in Core ML, it also got the extension killed on a real iPhone. A probe build died about 21 MB above baseline.

So we trained a smaller one: character and word n-grams with logistic regression. The weights are 1 MB. On 884 real test texts:

The filter only moves texts to Spam, so a few more false holds was the right trade for a model that actually runs. The best model on paper did not fit where it had to run.

We are honest about two caveats. We set the validation recall target after seeing one test result, so these scores are slightly optimistic. And "wrong number" investment openers slip past the text filter, which is why the payment hold is the backstop.

Our first dataset split leaked

A read-only Codex review found the same scam campaigns in both train and test, and a validation set that was 91% generated text. Our first scores were too good to be true.

We clustered texts by character 5-gram similarity, kept each cluster in one split, and made validation and test real texts only. The numbers above are from after that fix.

AI suggests, code decides money

One pure function decides if a payment pauses. This is decideHold from web/lib/rules.ts:

export function decideHold(input: {
  payment: PaymentFacts;
  holdOverCents: Cents;
  baseline: Baseline;
  guard: GuardSuggestion;
  circle?: CircleFacts;
}): HoldDecision {
  const { payment, holdOverCents, baseline, guard, circle } = input;
  const reasons = new Set<HoldReason>();

  if (circle?.frozen) reasons.add('frozen');

  const categoryReason = ALWAYS_HOLD[payment.category];
  if (categoryReason) reasons.add(categoryReason);

  if (payment.category === 'gift_cards' && circle?.atmCashRecently) reasons.add('cash_then_gift_card');

  if (!payment.payeeKnown) reasons.add('new_payee');

  // "Over $300" means strictly over: exactly $300.00 does not pause.
  if (payment.amountCents > holdOverCents) reasons.add('over_limit');

  if (isUnusual(payment.amountCents, baseline)) reasons.add('unusual_spending');

  if (payment.onCall) reasons.add('during_call');

  // The model's only power: adding a reason. Note there is no branch anywhere
  // in this function where it can remove one.
  if (guard && guard.score >= GUARD_HOLD_SCORE) reasons.add('guard_flagged');

  if (reasons.size === 0) return { action: 'allow' };
  return { action: 'hold', reasons: HOLD_REASONS.filter((r) => reasons.has(r)) };
}

Look at the last rule. The model's score can add a hold reason. No branch anywhere can remove one. A broken model can only make the system more careful, never less.

A language model does one other job: it explains a hold. Apple's on-device Foundation Models rewrites facts that code already picked into a calm note for the family. This came from a real bug. When we let the model explain a hold on its own, it wrote "the caller asked for gift cards" when there was no call. Now code picks only true facts, the model only rewords them, and if the note loses a risk fact, we send the plain facts instead.

Demo

• Live: www.checkwith.us. The "Try a payment" widget runs the same decideHold code in your browser.
• Video:

{This is a submission for the MLH x DEV Writing Challenge

Every scam script has the same line in it: "Don't tell your family."

So we built the opposite. checkwith.us is scam protection where the older adult does nothing new, and the family does the work. Before money moves, it checks with us.

We built it in 36 hours at SwampHacks XII, as a team of four. The code is public: github.com/builtbyd3v/checkwithus.

What I Built

Most scam tools ask the person at risk to make a decision under pressure. Scammers are very good at pressure. We moved the decision to the people who already worry about them.

The older adult keeps using their iPhone the same way:

• Texts. Our own scam model runs inside an iOS Message Filter extension, on the phone. Scam texts go to Spam, not the inbox.
• Calls. A Call Directory extension labels known scam numbers as "Possible scam".
• Payments. Every payment goes through fixed rules. A risky one pauses, and the phone says: "Your money is safe. Your family will call you."

The family gets the other half: approve or deny each held payment, see every reason in plain words, and freeze the account with one tap.

We set three hard rules before writing any code:

1. AI suggests, code decides money.
2. Money is integer cents everywhere.
3. The older adult's texts and bills never go to a cloud AI model.

The model had to fit where the texts are

The Message Filter is the only place on iOS that sees a text from an unknown sender. So the scam model had to run there, inside a very small memory budget.

Our first model was a fine-tuned DistilBERT. It scored well. At 67 MB in Core ML, it also got the extension killed on a real iPhone. A probe build died about 21 MB above baseline.

So we trained a smaller one: character and word n-grams with logistic regression. The weights are 1 MB. On 884 real test texts:

The filter only moves texts to Spam, so a few more false holds was the right trade for a model that actually runs. The best model on paper did not fit where it had to run.

We are honest about two caveats. We set the validation recall target after seeing one test result, so these scores are slightly optimistic. And "wrong number" investment openers slip past the text filter, which is why the payment hold is the backstop.

Our first dataset split leaked

A read-only Codex review found the same scam campaigns in both train and test, and a validation set that was 91% generated text. Our first scores were too good to be true.

We clustered texts by character 5-gram similarity, kept each cluster in one split, and made validation and test real texts only. The numbers above are from after that fix.

AI suggests, code decides money

One pure function decides if a payment pauses. This is decideHold from web/lib/rules.ts:

export function decideHold(input: {
  payment: PaymentFacts;
  holdOverCents: Cents;
  baseline: Baseline;
  guard: GuardSuggestion;
  circle?: CircleFacts;
}): HoldDecision {
  const { payment, holdOverCents, baseline, guard, circle } = input;
  const reasons = new Set<HoldReason>();

  if (circle?.frozen) reasons.add('frozen');

  const categoryReason = ALWAYS_HOLD[payment.category];
  if (categoryReason) reasons.add(categoryReason);

  if (payment.category === 'gift_cards' && circle?.atmCashRecently) reasons.add('cash_then_gift_card');

  if (!payment.payeeKnown) reasons.add('new_payee');

  // "Over $300" means strictly over: exactly $300.00 does not pause.
  if (payment.amountCents > holdOverCents) reasons.add('over_limit');

  if (isUnusual(payment.amountCents, baseline)) reasons.add('unusual_spending');

  if (payment.onCall) reasons.add('during_call');

  // The model's only power: adding a reason. Note there is no branch anywhere
  // in this function where it can remove one.
  if (guard && guard.score >= GUARD_HOLD_SCORE) reasons.add('guard_flagged');

  if (reasons.size === 0) return { action: 'allow' };
  return { action: 'hold', reasons: HOLD_REASONS.filter((r) => reasons.has(r)) };
}

Look at the last rule. The model's score can add a hold reason. No branch anywhere can remove one. A broken model can only make the system more careful, never less.

A language model does one other job: it explains a hold. Apple's on-device Foundation Models rewrites facts that code already picked into a calm note for the family. This came from a real bug. When we let the model explain a hold on its own, it wrote "the caller asked for gift cards" when there was no call. Now code picks only true facts, the model only rewords them, and if the note loses a risk fact, we send the plain facts instead.

Demo

• Live: www.checkwith.us. The "Try a payment" widget runs the same decideHold code in your browser.
• Video:

{https://www.youtube.com/watch?v=8C6eN3GvzcQ}

In the demo, Sam is a made-up older adult:

1. A scam text reaches Sam's phone. It goes to Spam. A scam number calls, and the screen says "Possible scam".
2. Sam takes $200 cash from an ATM, then tries to buy a gift card. The payment pauses. The family sees why: "Cash at an ATM, then a gift card". Each one alone is normal. Together, they are a classic scam.
3. The family app shows the amount, the payee and each reason. Approve asks for Face ID first. Deny does not, because it only stops money.
4. Freeze is one tap. One Nessie transfer moves the spending balance to a vault.

Partner Technologies

Capital One Nessie is the bank in every step. We use 10 resource types, and each one has a job: accounts for spending and a vault, merchants for payee categories, purchases, bills and withdrawals for every allowed payment, transfers for Freeze, and ATMs and branches for an alert when the phone is at a bank during a phone call.

The sandbox taught us things. It takes whole dollars only, so we refuse cents. Its reported balance never changes, so our ledger is the balance. And writes never retry: after an unknown outcome, we look the payment up by its tag, so a payment is never sent twice.

Auth0 (for AI Agents) handles login, roles and our bill-pay agent. The dashboard uses Auth0 login, and the iPhone app uses PKCE. Every family API route checks a signed access token. Risky actions like unfreeze, role changes and allowances need a sign-in from the last 5 minutes. Freeze does not, because it only protects money.

The bill-pay agent is its own Machine-to-Machine client with exactly one scope: pay:bills. Its bills go through the same rules as the phone, and every family route rejects machine tokens, so it can never approve a payment or freeze an account. On Oct 17 we verified it on the live tenant: it signed in with only pay:bills and paid a $350 water bill under the family's allowance.

Tiger Data holds the spending ledger. spending is a hypertable with columnstore compression and three real-time continuous aggregates: daily, weekly (built on daily) and per payee. The daily one powers the "much more than they usually spend" rule. A time-window query catches ATM cash followed by a gift card within 2 hours.

We measured it on 1,000,000 synthetic rows (200 families, 3 years) on Tiger Cloud:

Compression took the table from 163.3 MiB to 50.3 MiB (3.2x). The aggregates were fully materialized, so that is a best case.

GoDaddy Registry runs the .us domain, and the name is the pitch. Read it out loud: check with us. We registered it on Friday night, right after kickoff. It is the site, the iPhone app's API address and the Auth0 API audience.

Hackathon Experience

We were a team of four at SwampHacks XII (Oct 16 to 18, 2026), and we entered the Best First Timers track. Two of us, Hitesh and Rishon, were at their first hackathon. Hitesh wrote the pitch script and planned the demo video. Rishon did the design research that Dev turned into the app and dashboard design.

The biggest lesson came from a real phone. Our best model worked on a laptop and died in the extension. Testing on a real iPhone changed the whole ML plan.

The second lesson: test against the real service. Our Auth0 Guardian push passed every test against a fake Auth0. The real tenant rejected one parameter we always sent, so every push would have failed. We only found it because we ran it live.

We built with Claude Code and OpenAI Codex CLI. Claude Code wrote most of the code. Codex continued parts of the web backend and reviewed changes read-only. It found the data leak above and real payment bugs, like a retry that sent a new request instead of the same one. We checked every finding before we fixed it.

Next, we want to test the screens with older adults and their families, add server push alerts, and use a real scam-number feed for calls. Our scores describe our test set, not every scam.

Before money moves, check with us.
}

In the demo, Sam is a made-up older adult:

1. A scam text reaches Sam's phone. It goes to Spam. A scam number calls, and the screen says "Possible scam".
2. Sam takes $200 cash from an ATM, then tries to buy a gift card. The payment pauses. The family sees why: "Cash at an ATM, then a gift card". Each one alone is normal. Together, they are a classic scam.
3. The family app shows the amount, the payee and each reason. Approve asks for Face ID first. Deny does not, because it only stops money.
4. Freeze is one tap. One Nessie transfer moves the spending balance to a vault.

Partner Technologies

Capital One Nessie is the bank in every step. We use 10 resource types, and each one has a job: accounts for spending and a vault, merchants for payee categories, purchases, bills and withdrawals for every allowed payment, transfers for Freeze, and ATMs and branches for an alert when the phone is at a bank during a phone call.

The sandbox taught us things. It takes whole dollars only, so we refuse cents. Its reported balance never changes, so our ledger is the balance. And writes never retry: after an unknown outcome, we look the payment up by its tag, so a payment is never sent twice.

Auth0 (for AI Agents) handles login, roles and our bill-pay agent. The dashboard uses Auth0 login, and the iPhone app uses PKCE. Every family API route checks a signed access token. Risky actions like unfreeze, role changes and allowances need a sign-in from the last 5 minutes. Freeze does not, because it only protects money.

The bill-pay agent is its own Machine-to-Machine client with exactly one scope: pay:bills. Its bills go through the same rules as the phone, and every family route rejects machine tokens, so it can never approve a payment or freeze an account. On Oct 17 we verified it on the live tenant: it signed in with only pay:bills and paid a $350 water bill under the family's allowance.

Tiger Data holds the spending ledger. spending is a hypertable with columnstore compression and three real-time continuous aggregates: daily, weekly (built on daily) and per payee. The daily one powers the "much more than they usually spend" rule. A time-window query catches ATM cash followed by a gift card within 2 hours.

We measured it on 1,000,000 synthetic rows (200 families, 3 years) on Tiger Cloud:

Compression took the table from 163.3 MiB to 50.3 MiB (3.2x). The aggregates were fully materialized, so that is a best case.

GoDaddy Registry runs the .us domain, and the name is the pitch. Read it out loud: check with us. We registered it on Friday night, right after kickoff. It is the site, the iPhone app's API address and the Auth0 API audience.

Hackathon Experience

We were a team of four at SwampHacks XII (Oct 16 to 18, 2026), and we entered the Best First Timers track. Two of us, Hitesh and Rishon, were at their first hackathon. Hitesh wrote the pitch script and planned the demo video. Rishon did the design research that Dev turned into the app and dashboard design.

The biggest lesson came from a real phone. Our best model worked on a laptop and died in the extension. Testing on a real iPhone changed the whole ML plan.

The second lesson: test against the real service. Our Auth0 Guardian push passed every test against a fake Auth0. The real tenant rejected one parameter we always sent, so every push would have failed. We only found it because we ran it live.

We built with Claude Code and OpenAI Codex CLI. Claude Code wrote most of the code. Codex continued parts of the web backend and reviewed changes read-only. It found the data leak above and real payment bugs, like a retry that sent a new request instead of the same one. We checked every finding before we fixed it.

Next, we want to test the screens with older adults and their families, add server push alerts, and use a real scam-number feed for calls. Our scores describe our test set, not every scam.

Before money moves, check with us.