反向影片搜尋的運作原理與自建方法

Yes. Reverse video search exists, but the big search engines do not offer it as a single upload box. The usual workflow is to pull keyframes out of the video and run each one through a reverse image search. Under the hood, a dedicated system does the same thing at scale: it samples frames or detects shot boundaries, turns each frame into a perceptual hash or an embedding, stores them in an index, and matches the frames of your query clip against that index. The best-matching frames also carry timestamps, so the answer comes back as "this file, at 01:42". Consumer tools search the public web's image index. If you want to search a video library you own, you build or buy the index yourself.

Consumer options

Google Lens. Google's help page says you can search with an image by uploading a file, dragging it in, or pasting an image URL. Take a frame from the video, save it as an image, and upload it. The page does not mention video input, so the frame step is yours.

Bing Visual Search. Microsoft's support page describes searching the web with an image from the camera icon on Bing Images. It accepts uploads, pasted images and URLs. Same workflow: export a frame, upload it.

TinEye. TinEye fingerprints the image you submit and compares it with its index, which it builds by crawling the web. Its help center says it finds exact and altered copies, including cropped, color adjusted, resized, heavily edited or slightly rotated ones, and that it does not commonly return similar images of the same subject. Its help pages describe image search only.

InVID-WeVerify. This browser plugin was built for journalists and fact-checkers. Its keyframe tab takes a video URL or an uploaded file, segments it into keyframes, and lets you right-click a keyframe to search it on Google, Yandex, TinEye and Baidu. It automates the manual workflow above, so it is the quickest way to try it.

YouTube Content ID. Content ID does match video, but only for rights holders. Google's help page says owners must hold exclusive rights to a substantial body of original material that is frequently uploaded to YouTube. They supply reference files, and every new upload is scanned against that library. A regular user cannot submit a clip and ask where it appears.

What these tools miss

All four web options search an index of images. None of them has your video in it. That has practical consequences:

• A clip that circulated only as video, never as a thumbnail, screenshot or article image, may have no still in the index to match.
• You choose which frames to search. A frame from a title card or a black transition returns nothing useful, and a single frame says nothing about when in the clip it occurs.
• Copies that were cropped, mirrored, letterboxed, overlaid with captions or re-encoded at low quality produce frames that differ from the originals. TinEye's documentation covers crops and edits, but horizontal flips and heavy overlays are where fingerprint matching usually gives out, so test your own samples.
• Your private footage is not on the web, so no public tool can find it. Searching a library you own requires indexing it.

Build one over your own library

The pipeline has four steps.

1. Sample frames. The simplest approach is a fixed rate such as one frame per second. A more economical one is shot-based: ffmpeg's select filter exposes a scene score between 0 and 1, and the documentation says a threshold between 0.3 and 0.5 is generally a sane choice. Extract one representative frame per shot and store its timestamp.

2. Choose a fingerprint. Perceptual hashes (average, perceptual, difference, wavelet) from the imagehash library produce a short code, and the distance between two codes is a Hamming distance. They are cheap and handle re-encoding, resizing and mild color shifts. They are weak against heavy crops and flips. Embeddings from a vision model such as CLIP are larger and need a vector index, but they tolerate more edits and also support searching by meaning. Many systems run a hash for near-exact copies and embeddings for everything else.

3. Index. Store one row per sampled frame: video ID, timestamp, hash or vector. For hashes, a linear scan over a few million rows is fine. Past that, use a vector database or a BK-tree.

4. Align in time. Matching single frames gives noisy results, because unrelated frames sometimes collide. Real matches agree on an offset. If query frame at 3 s matches library frame at 63 s, and query frame at 4 s matches library frame at 64 s, the offset is 60 s in both cases. Count votes per (video, offset) and return the winner. That also gives you the start timestamp of the clip inside the source file.

Here is a small version using hashes and offset voting. It needs ffmpeg on your PATH, plus pip install imagehash pillow.

import subprocess, tempfile, glob
from collections import Counter
from PIL import Image
import imagehash

def frames(video, fps=1):
    """Return [(timestamp_seconds, phash)] sampled at `fps`."""
    with tempfile.TemporaryDirectory() as d:
        subprocess.run(
            ["ffmpeg", "-loglevel", "error", "-i", video,
             "-vf", f"fps={fps}", f"{d}/f_%06d.png"],
            check=True,
        )
        out = []
        for i, path in enumerate(sorted(glob.glob(f"{d}/f_*.png"))):
            out.append((i / fps, imagehash.phash(Image.open(path))))
        return out

# Build the index: one row per sampled frame.
index = []  # (video_id, timestamp, hash)
for vid in ["a.mp4", "b.mp4"]:
    index += [(vid, t, h) for t, h in frames(vid)]

def search(clip, max_dist=10):
    votes = Counter()
    for tq, hq in frames(clip):
        for vid, ts, h in index:
            if hq - h <= max_dist:          # Hamming distance
                votes[(vid, round(ts - tq))] += 1
    if not votes:
        return None
    (vid, offset), n = votes.most_common(1)[0]
    return vid, offset, n

print(search("query_clip.mp4"))  # e.g. ('a.mp4', 60, 14)

The third number is the vote count, which works as a confidence signal. The distance threshold of 10 on a 64-bit hash is a starting point, so tune it on clips you know are matches and clips you know are not. The first frame of the extracted images is labeled 0 seconds, which is approximate to within one sampling interval.

For production use, replace the linear scan with an index, add shot detection so static scenes do not produce hundreds of near-identical rows, and run a second embedding pass for mirrored or cropped copies.

Disclosure: I work on Mixpeek. Mixpeek indexes video in your own S3 or GCS bucket, splits it into scenes, and embeds frames. A retriever then returns the matching file and timestamp for an uploaded clip or image. We compare ten approaches in Best Reverse Video Search Tools, and the reverse search tutorial walks through the setup.