Heatmapify 重建 YouTube 影片的觀眾重播熱度圖

Hacktoberfest Open-Source AI Challenge: Ghost in the Machine Submission 👻🌼

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine

What I Built

Every YouTube video is haunted.

Thousands, sometimes millions, of people have watched it before you. You never see them, but they leave traces. They drag the playhead back to a moment and watch it again. They leave comments like "4:12 🤯". They drop off when the video drags. YouTube collects all of this into the "Most replayed" curve, the faint grey wave above the progress bar.

On a lot of videos, though, that curve isn't there. Newer videos, smaller channels and many long videos simply don't show it.

Heatmapify brings those viewers back. Paste a YouTube link and it tells you the most and least engaging moments of the video:

1. If YouTube publishes "Most replayed" data, Heatmapify reads it directly and marks it official with high confidence.
2. If it doesn't, Heatmapify builds an estimate from the traces people and the video leave behind:
   • 💬 Comments: every timestamp people mention ("3:42") becomes a bump on the curve
   • 📝 Captions: transcript windows are scored for hooks, emphasis, questions, claims and topic shifts
   • 🔊 Audio: loudness, onsets and spectral flatness
   • 🎞️ Visuals: motion and cut density from a low-res ffmpeg stream
3. If none of that is usable, it says no_data_available. It never makes up a curve.

Every estimate is labelled as one, with a confidence level, the signals used, and the signals skipped and why.

Why "Ghost in the Machine"? The heatmap is a record of past viewers: people no longer watching whose attention still shapes the video. When YouTube won't show that record, Heatmapify reconstructs it from what they left behind.

Demo

🔗 Live dashboard: https://heatmapify.onrender.com

• Click "See a sample result →" for a full result with no setup.
• To analyse your own links, run the open-source server on your Windows or Mac (one double-click, about 5 minutes) and connect the dashboard to it. YouTube blocks most cloud IPs, so this is how it stays reliable.

Code

velvetthunder15 / heatmapify

Extracts the most and the least engaging parts of any given Youtube video. Gives an elaborate estimation when 'Most Replayed' data is unavailable using comments, captions, audio and visuals.

Heatmapify

Runs on your own Windows or Mac for free. Paste a YouTube link, get the most and least engaging moments of the video, with a dashboard that shows exactly why.

• Uses YouTube's own "Most replayed" data when the video has it.
• Otherwise builds a clearly labelled estimate from four signals: comments, captions, audio and visuals.
• Never makes numbers up: if nothing usable exists, you get a clear error.

Dashboard: https://heatmapify.onrender.com Host your own copy:

Quick start: Windows · macOS

Quick start (Windows, about 5 minutes)

You need Windows 10 or 11 and nothing else. No Docker, no WSL, no Redis, no paid hosting.

1. Get the code

Either clone it:

git clone https://github.com/velvetthunder15/heatmapify.git
cd heatmapify

or…

MIT licensed. FastAPI + RQ + Redis (or single-process in-memory mode), yt-dlp, librosa, ffmpeg, plus a plain HTML/CSS/JS dashboard.

Timeline note: the repository was created on Oct 26 at about 01:00 PDT, roughly 8 hours before the challenge window opened at 09:00 PDT. The first two commits (the v1 API and a self-hosting script) came before 09:00 PDT. Everything else, including the dashboard, Render deploy, Windows/macOS runners, security hardening and score calibration, was built during the challenge week.

How I Built It

YouTube URL
   │
   ▼
official "Most replayed"? ──yes──▶ segments (source: official, confidence: high)
   │ no
   ▼
split the video into 10-second windows
   │
   ├─ comments   → timestamp density
   ├─ captions   → hook / emphasis / topic-shift scoring
   ├─ audio      → loudness / onsets / flatness (librosa)
   └─ visuals    → motion & cut density (ffmpeg frames)
   │
   ▼
weighted blend → reshaped to look like real "Most replayed" data → top & bottom moments

• Official data first. yt-dlp's heatmap field, then a fallback that parses ytInitialData from the watch page (old heatMarkers and newer macroMarkersListEntity).
• Jobs. Official lookups return at once; full estimates run as RQ background jobs while the dashboard shows progress. Results are cached (24 h official, 7 days estimates).
• Calibration. A reshaping step and a small calibrate tool make a typical estimated segment score like a typical official one.
• Optional LLM pass. Caption scoring can be blended with an LLM rating through any OpenAI-compatible endpoint, including a local open-weight model. It's off by default, and Heatmapify works fully without it.
• Honest output. Every response includes source, confidence, confidence_reason, signals_used and signals_skipped.

How I Used Render

Render is the public face of Heatmapify: it's where anyone opens it.

• The dashboard is a Render Static Site, defined as code. render.yaml is a Render Blueprint that publishes dashboard/ and auto-deploys on every push to main.
• Security headers live in the Blueprint. A strict Content-Security-Policy (only YouTube's image CDN and youtube-nocookie.com embeds), X-Frame-Options: DENY, nosniff, a referrer policy, a locked-down Permissions-Policy and COOP. Every deploy and every fork ships the same protections.
• One-click self-hosting. A Deploy to Render button in the README stands up your own copy from the Blueprint.
• Splitting the parts. The UI runs on Render; YouTube processing runs on hardware the user controls. The dashboard talks to their server over HTTPS with a session-only key, so nobody's history or key sits on a shared server.

Why Does Open Innovation Matter?

• It runs where YouTube lets it. A closed SaaS version would spend its life fighting captchas. Open source means anyone can run it on their own connection, free.
• The scoring isn't a black box. Weights, calibration and keyword lists are plain config and plain Python. Think comments should count more than visuals on gaming videos? Change WEIGHT_COMMENTS and see.
• It's built on open shoulders: yt-dlp, youtube-transcript-api, librosa, ffmpeg, FastAPI, RQ and Redis. Heatmapify is MIT licensed, so the next person can take it further.

Prize Categories

• Best Use of Render: dashboard deployed from a Render Blueprint with auto-deploys, security headers in code, and a one-click Deploy to Render button.