如何以 OpenCLIP 和 NPU 建立零雲端照片去重工具

Every new laptop today is marketed as an ‘AI PC’ thanks to a built-in Neural Processing Unit (NPU). But beyond chatting with small local LLMs, what practical work can this silicon actually do? Recently, I hit a wall with a classic IT problem: data gravity. I had about a hundred gigabytes of scanned family photos that needed to be organized and enhanced. Uploading gigabytes of media to a cloud AI API for something as straightforward as deduplication is unreasonable. The compute needed to move to the data, and that meant putting my laptop’s idle NPU to work. Source code for the scripts used in this article is available on GitHub.

Archiving thousands of pre-digital-age photos and massive video files is a common challenge for most households. Those archives easily span terabytes, making cloud uploads a bandwidth nightmare. While data privacy is a major reason to keep workloads local, data gravity is the physical bottleneck I had to design around here.

It definitely isn’t just my problem—it’s the industry’s. The default architecture of the AI boom, which ships every inference task to a centralized cloud API, is colliding with hard physical limits. The electrical grid simply cannot support the current pace of hyperscale build-outs. Recent industry projections show US utility capacity falling short of data center power demand by over 100 gigawatts by 2030. Regulators are actively stepping in to throttle this: just days ago in September 2026, Virginia (the densest data center region in the world) signed Executive Order 22, heavily restricting state assistance for new hyperscale deployments over 25 megawatts due to acute grid and water constraints. The cloud’s physical room is running out, which is exactly why compute has to move to the data.

Fortunately, consumer hardware is evolving to support this. Most modern machines now ship with an NPU specifically designed for tensor operations. Let’s look at how edge processing can replace cloud APIs for heavy media.

Understanding the edge hardware trade-offs

To understand why local AI architecture is shifting, it helps to look at how modern silicon divides the workload across different classes of hardware:

• CPU (Central Processing Unit): It handles system logic and file I/O effortlessly, but running continuous neural network math requires brute force—pegging core usage at 100%, drawing 35W+ of power and rapidly draining a laptop battery.
• GPU (Graphics Processing Unit): Built for massive graphics rendering and high-throughput compute, but highly power-hungry and often reserved for heavy graphics or dedicated model running or training.
• NPU (Neural Processing Unit): Purpose-built specifically for low-power matrix operations on consumer devices, it handles smaller AI workloads at single-digit wattage (~6W).
  Apple platforms are a special case. Projects like qwen-ane are proving that Apple’s 32-core NPU can run massive 27B models while sipping just 8W of power. While this level of efficiency is exactly what the industry needs, it is still strictly experimental—the inference speeds aren’t yet viable for production workloads.
  High-end AI desktops like the NVIDIA GB10 also don’t have NPUs. Instead, they rely on massive GPUs packed with thousands of Tensor Cores and unified memory, with combined consumption sometimes exceeding 160W.

Building a semantic deduplication pipeline with OpenCLIP

For this project, I focused on image deduplication and making the idle NPU speed up the pipeline.

Very simply put, semantic deduplication works by comparing the distance between image embeddings—high-dimensional digital representations of the pictures. Generating these embeddings is the heavy lifting done by the AI model. Before anything reaches the model, each photo goes through a pinned preprocessing step: resize the short side to 256 pixels, center-crop to 224 and then normalize with OpenAI’s published constants. Applying the exact same crop and transform to every image on every run is what keeps the comparisons mathematically stable.

The foundation for this approach was laid in March 2021, when OpenAI introduced Contrastive Language-Image Pre-training (CLIP). Prior to CLIP, computer vision models were largely rigid classifiers trained on fixed datasets like ImageNet, capable of recognizing only a pre-defined set of categories. OpenAI changed this by training a dual-encoder architecture on 400 million image-text pairs scraped from the web. By training an image encoder and text encoder simultaneously to pull matching pairs together in a shared vector space and push mismatched pairs apart, CLIP learned generalized visual concepts rather than narrow labels.

When OpenAI released CLIP, they published the pre-trained weights for several architectures, including ResNet variants and a then-novel Vision Transformer (ViT). Over the last few years, the open-source community (notably the OpenCLIP project) and other AI labs have scaled this architecture massively, releasing much heavier variants like ViT-L (Large), ViT-H (Huge), and ViT-g (Giant).

I chose the OpenCLIP implementation of OpenAI’s original ViT-B-32-quickgelu model for this pipeline as it sits at the exact sweet spot for edge hardware: small enough to compile directly into NPU memory without thrashing the host system, yet sophisticated enough to recognize that two photos taken from slightly different angles share the exact same subject. To generate these embeddings locally, my stack relies on Python 3.14, PyTorch, OpenCLIP ViT-B/32 and ImageMagick for parallel preprocessing. When it comes to model compilers, the architecture gets interesting, as each vendor has their own implementation. On the Intel platform, I used OpenVINO—Intel’s model compiler that converts the PyTorch encoder into an optimized Intermediate Representation (IR) capable of running on any of the machine’s accelerators: CPU, GPU or NPU.

python3.14 -m venv .venv
source .venv/bin/activate

pip install --upgrade pip

# PyTorch — the CPU wheels are all this pipeline needs on the host side
pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu

# The model: OpenCLIP ships OpenAI's pre-trained ViT-B-32 weights
pip install open_clip_torch

# Intel's model compiler + runtime: converts the encoder to an IR
# that the NPU can execute
pip install openvino

# The ONNX toolchain: torch exports the encoder to ONNX,
# which OpenVINO then converts into an IR
pip install onnx onnxscript

# Host-side image handling
pip install numpy pillow

# Parallel preprocessing — embed.py auto-detects this
# Linux:   sudo apt-get install -y imagemagick
# macOS:   brew install imagemagick
# Windows: winget install ImageMagick

Terminal commands to install the required Python 3.14 stack, including OpenCLIP and Intel’s OpenVINO runtime.

Benchmarking the NPU against the CPU with OpenVINO

When I ran the initial PyTorch script on the CPU, embedding generation was highly taxing. My baseline throughput hovered around 35 milliseconds per photo and power consumption hitting 47W. While that might sound fast for a single image, processing a massive archive kept my Intel Core Ultra’s CPU pegged at nearly 100% utilization. The laptop fans spun up immediately, and system monitoring tools showed a power draw exceeding 46 watts just to crunch through the matrix math. It was a brute-force approach that monopolized the machine and rapidly drained the battery.

$ sudo ./.venv/bin/python embed.py ./scan --backend cpu

preprocessing: /usr/sbin/magick (parallel, 4 workers)
prepared 373 crop(s) in 0.0s
[1/373] photo0000.png     35.3 ms -> photo0000.vitb32.npy
[2/373] photo0002.png     32.9 ms -> photo0002.vitb32.npy
[3/373] photo0020.png     35.2 ms -> photo0020.vitb32.npy
…
embedded 373 in 13.3s (28.0 img/s) [backend=cpu, batch=1], 0 skipped (cached), avg power 46.9 W

Running the OpenCLIP model purely on the CPU monopolizes resources and averages ~35 milliseconds per image and 47W of power consumption. Sudo is required for power stats access.

To fix this, I exported the model to OpenVINO’s Intermediate Representation (IR) format and routed the workload through Intel’s Level-Zero driver API directly into the NPU. One gotcha worth knowing: the ONNX exporter bakes the batch size into the graph’s internal reshape constants, so the script exports one IR per batch size and the runtime loads whichever matches –batch.

My first pass used a naive, synchronous loop—sending one image down to the silicon at a time. This yielded a modest 1.5x speedup, and impressive 2x lower power consumption. The bottleneck wasn’t the NPU itself, but rather the host-device dispatch overhead. The NPU finished the tensor math in milliseconds and then sat idle, waiting for Python to hand it the next image. (As a quick aside: the Level-Zero driver compiles a hardware blob on the first execution, which causes a brief delay, but subsequent runs initialize instantly. Once compiled, the encoder runs in fp16, which halves its memory traffic on the silicon).

$ python export_ov.py --batch 1

exporting image encoder to OpenVINO IR (static batch=1) ...
IR: ~/photo-dedup/dedup/models/clip-vitb32-b1.xml
input : [1,3,224,224] <Type: 'float32'>
output: [1,512] <Type: 'float32'>

The one-time export bakes the PyTorch encoder into a static-batch OpenVINO IR—one build per batch size, loaded by the NPU runs that follow.

$ sudo ./.venv/bin/python embed.py ./scan --backend npu

backend: NPU (compiled clip-vitb32-b1.xml, hint=LATENCY, batch=1)
preprocessing: /usr/sbin/magick (parallel, 4 workers)
prepared 373 crop(s) in 0.0s
[1/373] photo0000.png     26.1 ms -> photo0000.vitb32.npy
[2/373] photo0002.png     25.6 ms -> photo0002.vitb32.npy
[3/373] photo0020.png     24.9 ms -> photo0020.vitb32.npy
…
embedded 373 in 5.7s (65.3 img/s) [backend=npu, batch=1], 0 skipped (cached), avg power 27.3 W

A naive, synchronous NPU loop improves throughput to ~25 milliseconds per image, but leaves the accelerator idle between dispatch calls. Power consumption significantly decreases to 27W

To unlock the silicon’s true potential, I restructured the script to use batched inference and applied OpenVINO’s THROUGHPUT performance hint. By feeding the NPU batches of 16 images simultaneously, I saturated its execution units and eliminated the dispatch bottleneck. Per-photo inference time dropped from ~36 ms on the CPU to about 13 ms on the NPU, and end-to-end throughput climbed from roughly 28 photos per second (a serial, single-image pass) to over 92—a 3x win. But raw speed wasn’t the real victory—efficiency was. The NPU silently chewed through thousands of images in the background, drawing a mere 22W, allowing processing of a massive media archive on battery power without triggering the fans.

$ python export_ov.py --batch 16

exporting image encoder to OpenVINO IR (static batch=16) ...
IR: ~/photo-dedup/dedup/models/clip-vitb32-b16.xml
input : [16,3,224,224] <Type: 'float32'>
output: [16,512] <Type: 'float32'>

PyTorch encoder export for batch size 16.

$ python embed.py ~/scan --backend npu --batch 16

backend: NPU (compiled clip-vitb32-b16.xml, hint=THROUGHPUT, batch=16)
preprocessing: /usr/sbin/magick (parallel, 4 workers)
prepared 373 crop(s) in 0.0s
[1/373] photo0000.png     10.8 ms -> photo0000.vitb32.npy
[2/373] photo0002.png     10.8 ms -> photo0002.vitb32.npy
[3/373] photo0020.png     10.8 ms -> photo0020.vitb32.npy
…
embedded 373 in 4.0s (92.6 img/s) [backend=npu, batch=16], 0 skipped (cached), avg power 22.7 W

Processing images in batches of 16 using OpenVINO’s THROUGHPUT hint cuts per-photo inference time to roughly 13 milliseconds while drawing minimal power.

Filtering the noise with cosine similarity

Now I proceed to the actual goal: detecting the duplicate photos. To reiterate, I assume there are no exact binary duplicate images in the collection for this step.

The script reads the 512-dimensional embeddings generated by OpenCLIP and normalizes each to unit length, which turns “cosine similarity” into a plain dot product—one matrix multiply across the whole archive. By calculating the similarity between these vectors—essentially measuring how aligned two points are in mathematical space—the script gets a numerical score showing how visually similar two photos are.

Photos that clear the similarity threshold (I use 0.92) get grouped into a “duplicate cluster.” Because similarity is chained transitively through union-find, a burst of five shots where each neighbor just barely matches is still pulled into a single cluster. Each cluster’s photos—and their embedding sidecars—are moved together into a dup_XXXX/ folder, while unique photos stay put. The scan is non-recursive into subfolders, so re-running the tool on an already-sorted archive is a safe no-op; a –dry-run flag identifies the clusters first without moving anything.

$ python cluster.py ~/scan

2 cluster(s), 9 photo(s), threshold 0.92:
  dup_0001/ (5)
    IMG_0412.jpg
    IMG_0413.jpg
    IMG_0414.jpg
    IMG_0415.jpg
    IMG_0416.jpg
  dup_0002/ (2)
    IMG_0788.jpg
    IMG_0789.jpg
moved 9 photo(s) into 2 cluster folder(s)

The final pass compares the NPU-generated vectors using cosine similarity, groups every cluster above the threshold and moves each duplicate group into its own folder.

What local NPUs mean for the future of edge compute

I have demonstrated how a lightweight model can tap into the NPU sitting inside a standard modern laptop to solve a real, practical problem in the most energy-efficient way. Batching embeddings for semantic deduplication is a great fit for local silicon, but it only scratches the surface of what edge hardware can do. In my next articles, I’ll look at running heavier, more complex models locally to tackle a variety of practical tasks.