ScopeGuard：我如何打造符合 NDA 保密要求的本地 Gemma 3 智能體，為自由工作者守住 $1,190 收入

This is a submission for the Hacktoberfest Open-Source AI Challenge: Week 2 ("Side Hustle")

What I Built: ScopeGuard

Every freelance developer and boutique agency owner knows the dreaded Friday afternoon Slack message that starts with four seemingly innocent words:

"Hey! Quick thought for launch..."

Behind those five words usually hides 14 to 20 hours of uncontracted architectural work—a "lightweight" role-based admin panel, an extra Stripe webhook flow, or dark-mode theming across 12 screens.

Freelancers face a painful dilemma:

1. Eat the hours and watch your effective hourly rate plummet.
2. Push back awkwardly and risk sounding defensive or damaging a warm client relationship.

ScopeGuard is a $19/month micro-SaaS and contract shield built for independent developers, designers, and consultants. Before replying to a client's "quick extra request," you drop their message into ScopeGuard. In under 800 milliseconds, an air-gapped Mastra agent retrieves your signed Statement of Work (SOW) clauses from MongoDB Atlas, evaluates the request using a Tinker fine-tuned Google Gemma 3 4B model, calculates the exact unbilled revenue at risk, and drafts a diplomatic, ready-to-send Change Order email—complete with itemized line items and a 1-click approval quote.

Why Open-Source AI Beats Closed APIs Here

For a tool auditing legal contracts and unreleased product specifications, closed frontier APIs (OpenAI GPT-4o, Anthropic Claude 3.5) fail on three fundamental business requirements:

1. Strict Client NDA Compliance (Zero Cloud Data Leakage)

Freelancers and agencies sign strict Non-Disclosure Agreements (NDAs). Pasting a client's proprietary FinTech architecture, hourly rates, and signed SOW clauses into a third-party cloud API violates enterprise confidentiality clauses. By running open-weight Google Gemma 3 4B locally (or on an isolated private instance), zero bytes of client IP ever leave your infrastructure.

2. Niche Fine-Tuning Beats "Polite" Base Models

Out-of-the-box frontier LLMs are RLHF-trained to be agreeable assistants. When asked to evaluate a client message, base models tend to summarize the request rather than aggressively hunting for contractual boundary violations. Using Tinker, I fine-tuned a LoRA adapter (Rank 16, Alpha 32) on Gemma 3 4B specifically trained to detect conversational scope traps ("should just be a simple toggle"), cite exact SOW exclusions, and output deterministic JSON schemas.

3. Unit Economics That Keep a $19/mo Micro-SaaS Profitable

If a boutique agency connects ScopeGuard to their Slack workspace to audit 10,000 incoming client messages and SOW clauses per month, token taxes on closed frontier models destroy SaaS margins:

Demo

1. Live Scope Auditor Dashboard

When a client asks for a "quick role-based admin portal before launch," ScopeGuard flags it as OUT_OF_SCOPE_CREEP (Risk Score: 96/100), cites Section 2.1 - Deliverables & Exclusions, calculates $1,190 in Unpaid Revenue Protected (14 extra hours @ $85/hr), and generates a polite Change Order email:

2. Sentry Distributed AI Agent Tracing

Every audit run is instrumented with @sentry/nextjs OpenTelemetry spans (gen_ai.invoke_agent $\rightarrow$ gen_ai.execute_tool $\rightarrow$ gen_ai.generate_content), capturing exact millisecond latencies, token throughput, and protected revenue metrics:

System Architecture & Partner Stack

ScopeGuard stacks 5 partner technologies into a cohesive, bloat-free pipeline:

flowchart TD
    ClientMsg["Client Request (Slack / Email)"] --> UI["Next.js 16 Web Dashboard"]
    UI --> API["/api/analyze (App Router)"]
    API --> SentrySpan["Sentry Distributed Tracing (gen_ai spans)"]
    SentrySpan --> Mastra["Mastra AI Agent Engine"]
    Mastra --> Tool["sowRetrieverTool"]
    Tool --> Mongo[("MongoDB Atlas SOW Collection")]
    Mongo --> Tool
    Tool --> Mastra
    Mastra --> Gemma["Fine-Tuned Gemma 3 4B (Tinker LoRA)"]
    Gemma --> AuditResult["Scope Verdict + Itemized Quote + Pushback Email"]
    AuditResult --> UI

How Each Partner Tool Powers the Workflow:

• Google Gemma 3 4B ($200 Tier): Serves as our local, open-weight reasoning engine via an OpenAI-compatible Ollama/vLLM endpoint (@ai-sdk/openai), guaranteeing 100% NDA privacy.
• Tinker ($200 Tier): Used to fine-tune Gemma 3 4B (scripts/generate-tinker-dataset.mjs & scripts/evaluate-tinker.mjs) on freelance SOW clauses vs. conversational scope-creep scenarios.
• Mastra (@mastra/core) ($100 Tier): Orchestrates scopeGuardAgent and executes deterministic tool calls (sowRetrieverTool) to ground every audit in the project's actual signed contract.
• MongoDB Atlas ($100 Tier): Stores active client SOW documents (scopeguard.sow_clauses), hourly billing rates, NDA levels, and strict exclusion boundaries—paired with an automatic in-memory fallback (src/lib/contracts.ts) for instant zero-config local evaluation.
• Sentry Agent Tracing ($100 Tier): Wraps the entire Mastra + Gemma execution pipeline in OpenTelemetry gen_ai.* spans, tracking gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, tool latency, and custom business telemetry (scopeguard.revenue_protected_usd).
• Render ($200 Tier): Configured via a production-ready render.yaml Blueprint for one-click cloud deployment.

Model Fine-Tuning & Evaluation: Base Gemma 3 vs. Tinker LoRA

A generic model often falls for polite phrasing like "Since the styles are already done, shouldn't take more than 10 minutes, right?"

To solve this, I built a domain-specific training dataset (scripts/tinker-dataset.jsonl) and evaluation benchmark (scripts/evaluate-tinker.mjs) testing 100 real-world freelance client messages across three classes: OUT_OF_SCOPE_CREEP, AMBIGUOUS_TRAP, and IN_SCOPE.

Tinker Benchmark Results (node scripts/evaluate-tinker.mjs)

Inside the Code: Instrumenting Mastra + MongoDB + Sentry

One of my favorite engineering parts of ScopeGuard is how cleanly Mastra tools and Sentry AI Agent Tracing compose together. Here is how sowRetrieverTool fetches contract boundaries from MongoDB Atlas while recording a custom gen_ai.execute_tool telemetry span:

// src/mastra/tools/sow-retriever.ts
import { createTool } from "@mastra/core/tools";
import { z } from "zod";
import * as Sentry from "@sentry/nextjs";
import { getProjectSow } from "@/lib/mongodb";

export const sowRetrieverTool = createTool({
  id: "retrieve-sow-clauses",
  description: "Retrieves signed Statement of Work (SOW) clauses and hourly rates from MongoDB Atlas.",
  inputSchema: z.object({
    projectId: z.string(),
    keywords: z.string(),
  }),
  execute: async ({ context }) => {
    return await Sentry.startSpan(
      {
        op: "gen_ai.execute_tool",
        name: "retrieve-sow-clauses (MongoDB Atlas)",
        attributes: {
          "gen_ai.tool.name": "retrieve-sow-clauses",
          "gen_ai.tool.input": JSON.stringify(context),
        },
      },
      async (span) => {
        const sowDoc = await getProjectSow(context.projectId);
        span.setAttribute("gen_ai.tool.output", JSON.stringify(sowDoc));
        return sowDoc;
      }
    );
  },
});

And inside /api/analyze/route.ts, we wrap the entire agent execution in a root gen_ai.invoke_agent span and attach custom business metrics so we can track total freelance revenue saved right inside Sentry:

agentSpan.setAttribute("gen_ai.usage.input_tokens", usage?.promptTokens ?? 0);
agentSpan.setAttribute("gen_ai.usage.output_tokens", usage?.completionTokens ?? 0);
agentSpan.setAttribute("scopeguard.revenue_protected_usd", parsedAnalysis.revenueAtRisk);

Testing ScopeGuard with a Real First User

To put ScopeGuard through a real-world stress test, I sat down with a fellow full-stack developer friend (Rakshitha) who regularly collaborates on web application builds and handles shifting feature requests.

The Test Scenario

We ran three real-world client messages through ScopeGuard against two different contract templates (FinTech Mobile Dashboard Redesign @ $85/hr and Headless Shopify Storefront Migration @ $95/hr):

1. Trap 1: A client asking to "quickly add a lightweight web admin portal for user role management before launch" $\rightarrow$ Flagged OUT_OF_SCOPE_CREEP ($1,190 protected).
2. Trap 2: A client requesting "Stripe Sandbox test mode with 4 country checkout variations by Friday" $\rightarrow$ Flagged OUT_OF_SCOPE_CREEP ($1,360 protected).
3. Control Test: A client asking to "tweak the primary button color on the Login screen from #0052FF to #0047E0" $\rightarrow$ Correctly passed as IN_SCOPE ($0 fee, covered under Section 4.5 UI Revisions).

First User Feedback & Iteration

"Normally when someone asks for an 'easy' admin view or an extra integration mid-sprint, I know it's out of scope, but writing the reply takes 20 minutes of second-guessing my tone. What actually sold me here wasn't just the risk score—it was seeing the hours broken down into separate line items inside the email. It makes the price feel objective rather than personal."

What I Changed Based on Her Feedback:
Initially, ScopeGuard only output a single lump-sum dollar figure ($1,190). Based on our test session, I updated the Gemma JSON output schema and UI to generate itemized Change Order line items (e.g., breaking the $1,190 quote into 9 hrs for Web Admin Role-Based Views [$765] + 5 hrs for Audit Log Table & QA [$425]) and added a 1-click .txt Change Order Download button so freelancers can attach a formal amendment directly to their email reply.

Try It Out & Repository Link

🔗 GitHub Repository: https://github.com/kishorekrrish3/ScopeGuard

Run ScopeGuard Locally in 60 Seconds

ScopeGuard includes a built-in resilient fallback mode—meaning you can clone and test all three preset scenarios, inspect the live OpenTelemetry trace waterfall, and run the Tinker evaluation benchmark right out of the box:

# 1. Clone and install dependencies
git clone [https://github.com/kishorekrrish3/ScopeGuard.git](https://github.com/kishorekrrish3/ScopeGuard.git)
cd scopeguard
npm install

# 2. Start the Next.js 16 development server
npm run dev

# 3. Optional: Seed MongoDB Atlas with SOW contracts
curl -X POST http://localhost:3000/api/seed

# 4. Run the Tinker LoRA vs. Base Gemma 3 evaluation suite
node scripts/evaluate-tinker.mjs

Thanks for reading—and here's to never building an unpaid "quick 5-minute admin portal" again! 🛡️