跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-10-12AI 評分41

VEDJE: Video-Efficient Discriminative Joint Encoder for Scalable Video-Text Retrieval

VEDJE: Video-Efficient Discriminative Joint Encoder for Scalable Video-Text Retrieval

AI 導讀

VEDJE 將影片壓縮成小型、具時間結構的快取,再以 33M 參數的聯合編碼器進行影片文字檢索。它在 MSR-VTT 取得 59.8 R@1,線上參數量比 LamRA 少約 49×;每段影片的快取為 48 KiB,較原始視覺特徵小 128.5×。保留壓縮後的時間結構有助檢索,特徵變化預測目標亦能提升檢索表現,且不增加推理時計算。

正文

🚀 Can we get the accuracy of a 7.6B-parameter video-language model using a much smaller retrieval system?

Introducing VEDJE: efficient video-text retrieval with compact, reusable video representations.

Rather than repeatedly processing full video features, VEDJE compresses each video once into a tiny, temporally structured cache. A lightweight 33M-parameter joint encoder then matches queries against the cached candidates.

The results:
📈 59.8 R@1 on MSR-VTT, matching LamRA with ~49× fewer online parameters.
📦 48 KiB per video, 128.5× smaller than the original visual features.
⚡ +3.6 to +7.2 R@1 points over matched first-stage retrievers on MSR-VTT.

The interesting finding: preserving temporal structure during compression matters. We also introduce a feature-change prediction objective that improves retrieval without extra inference-time computation.

🔗 Interactive project page: https://gabrieleserussi.github.io/vedje/

💻 Open-source implementation: https://github.com/GabrieleSerussi/vedje

▶️ Google Colab: https://colab.research.google.com/github/GabrieleSerussi/vedje/blob/main/colab/vedje_tour.ipynb

We're curious: could compact, reusable video representations become a practical foundation for large-scale multimodal search?

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co