Resume Scorer：本地優先的開源 AI 履歷評分工具

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

Resume Scorer — a local-first, open-source resume screening pipeline that turns a PDF into a structured, evidence-backed score in under two minutes.

Drop in a resume PDF, point it at a role, and get:

• Category scores with evidence — open source contributions, self projects, production experience, technical skills, AI fluency
• Bonus points — GSoC, portfolio site, startup founder history, LinkedIn presence
• Deductions — no project links, tutorial-only projects, unverifiable claims
• Key strengths and areas for improvement in plain English

Who would pay for this: any small company or startup drowning in junior applications. A 10-person startup posting a junior engineering role on LinkedIn can easily get 300–500 applicants in a week. Nobody has time to read all of those carefully. Resume Scorer runs the first pass locally, ranks the pile, and surfaces the top candidates for a human to actually read.

It also works for freelance recruiters who handle screening for multiple clients — they can define a custom role config per client, run it on their own machine, and deliver faster, more consistent results.

The scoring rubric lives entirely in a role.json file. Adding a new role requires zero Python — just run python score.py --init-role <name> and edit the scaffolded config.

$ python score.py resumes/sample.pdf --role software_engineering_intern

================================================================================

📊 RESUME EVALUATION RESULTS FOR: Jane Doe

🎯 OVERALL SCORE: 74.0/110

📈 DETAILED SCORES:

🌐 Open Source: 18/35
Evidence: 3 merged PRs to a popular OSS project (800+ stars), active
GitHub with 40+ repos, Hacktoberfest participant for 2 years.

🚀 Self Projects: 22/30
Evidence: Production RAG chatbot with LangChain, deployed on Vercel with
live demo. Two other projects lack links (-4 pts deducted).

🏢 Production Experience: 16/25
Evidence: 6-month SWE internship at a Series B startup. Shipped two
features to production. No founder or early-engineer signal.

💻 Technical Skills: 8/10
Evidence: Python, TypeScript, Go, PostgreSQL, Redis, Docker, Kubernetes.
Breadth demonstrated across multiple projects and work history.

🤖 AI Fluency: 10/10
Evidence: Built and deployed a RAG system with structured outputs,
embedding search, and hallucination guardrails. Describes
evaluation methodology and production monitoring setup.

⭐ BONUS POINTS: 2
Portfolio website: +2 | All other categories: +0

✅ KEY STRENGTHS:

1. Strong AI/ML engineering with production RAG deployment
2. Broad technical stack across backend, infra, and frontend
3. Consistent open-source activity with external contributions

🔧 AREAS FOR IMPROVEMENT:

1. Add live demo links to remaining projects

2. Increase depth of contributions to larger open-source projects

In development mode the tool also appends a CSV row per run, making it straightforward to sort and compare a full batch of candidates in a spreadsheet.

---

## Code

{% embed https://github.com/srikar-naidu/resume-scorer.git %}

---

## How I Built It

The entire pipeline runs on open-weight models via **Ollama** or any OpenAI-compatible endpoint. No closed API is required.

### Open-source stack

| Layer | What it does |
|---|---|
| **Ollama + Gemma** | Local inference. `gemma4:latest` is the default and runs on a laptop with 16 GB RAM |
| **PyMuPDF / pymupdf4llm** | Converts PDF pages to clean Markdown before any LLM call |
| **Jinja2** | All prompts are templates rendered from `role.json` at runtime — no hardcoded rubric text anywhere in the codebase |
| **Pydantic** | The LLM response is validated against a dynamically-built schema derived from the role config. Invalid point values are rejected before they reach the output |
| **GitHub API** | Enriches the resume with real contributor data, classifying repos as `open_source` (multiple contributors) vs `self_project` (single contributor) |

### Architecture

`score.py` is the CLI entrypoint. It calls:

1. `core/pdf.py` — extracts a structured `JSONResume` object section-by-section, each section getting its own LLM call with a Jinja template and a Pydantic schema for structured output
2. `core/github.py` — fetches and classifies repositories if the resume has a GitHub link
3. `core/evaluator.py` — assembles everything into a single text block and calls the LLM with the role's evaluation prompt, fully generated from `role.json`

The scoring schema is built dynamically with `pydantic.create_model` at runtime, so it reflects whatever categories the role defines. Adding a new role is a JSON edit — no Python required.

### Project layout

resume-scorer/
├── score.py # CLI entrypoint
├── roles.py # Role dataclass, load_role(), scaffold_role()
├── core/ # All processing logic
│ ├── models.py # Pydantic schemas + dynamic scoring models
│ ├── evaluator.py # LLM evaluation pipeline
│ ├── pdf.py # PDF → structured JSON extraction
│ ├── github.py # GitHub enrichment
│ ├── transform.py # JSON normalisation + text serialisation
│ └── llm_utils.py # Provider init + JSON extraction helpers
├── app_config/ # Configuration
│ ├── config.py # Provider resolution
│ └── providers.json # LLM provider registry
├── prompts/ # Jinja templates
│ └── templates/
├── roles/ # One subdirectory per role
│ └── software_engineering_intern/
│ ├── role.json # Rubric lives here
│ ├── criteria.jinja # Data-driven — loops over role.json
│ └── system_message.jinja
└── resumes/

---

## Why Does Open Innovation Matter?

Three concrete reasons this couldn't work the same way with a closed API:

**1. Candidate data stays on your machine.**

Resumes contain names, phone numbers, addresses, education history, and employment records. Many candidates have no idea their PDF might be forwarded to a third-party model provider. Running locally means none of that data leaves the recruiter's machine. This matters legally (GDPR Article 22 has explicit requirements around automated decision-making) and practically — a small company can't afford a data breach from a screening tool.

**2. The unit economics only work locally.**

Screening 500 applicants with a paid API at typical pricing costs $20–60 per batch in tokens alone, before GitHub enrichment calls. At volume, that's a real line item that makes the service unviable for small teams. On Ollama with a local model, the marginal cost per run is zero. The only cost is the hardware you already own.

**3. The rubric is yours to inspect and own.**

A closed screening API is a black box. You can't see why a candidate scored 62 vs 71, can't tune the weights, can't add a bonus for a skill that matters to your specific team. Because both the model and the prompt are open and local, every scoring decision is fully auditable — the `evidence` field in the output tells you exactly what the model saw and why it scored what it scored. When a candidate disputes a result, you can answer them with specifics.

Open-weight models made the privacy and cost arguments viable. The open-source tooling around them — Ollama's API compatibility, Pydantic's schema enforcement, Jinja's prompt templating — made the config-driven rubric design possible. A closed API can't give you either.

---

## Prize Categories

**Gemma — Best Use of Gemma**
`gemma4:latest` is the default model shipped in `app_config/providers.json`. The tool is built and tested against Gemma running locally through Ollama. The entire PDF extraction pipeline (six separate structured-output LLM calls per resume) and the final evaluation call all run against Gemma by default. Switching to a different model is a one-line change in `.env` — no code changes required.

**GitHub — Best Use of GitHub Copilot**
This project was built in VS Code with copilot and Kiro (an AI-powered IDE built on VS Code / GitHub Copilot infrastructure). The agentic coding workflow — spec, design, implement, verify in a loop — is directly visible in the session history. Kiro handled the config-driven refactor (making the rubric fully data-driven from `role.json`), the folder restructure, import rewrites across the whole codebase, and all verification runs, autonomously.