AVC-Reward:以事件為中心的音訊與影片一致性評估
AVC-Reward: Event-Centric Audio-Video Consistency Evaluation
AVC-Reward提出以事件為中心的音訊與影片一致性評估框架,並實作為可訓練的獎勵模型,用於評估複雜多事件場景。框架衡量整體敍事協調、事件層級時間同步及細緻屬性對齊。它與人類判斷的吻合度在自動評估方法中達到當前最佳,亦可定位失效原因,並作為線上強化學習訊號提升聯合生成模型的音訊與影片一致性。
Published on Oct 15
Authors:
,
,
,
,
,
,
,
Abstract
An event-centric framework evaluates audio-video consistency across complex multi-event scenes by measuring global harmony, temporal synchronization, and attribute alignment via a trainable reward model.
As joint audio-video generation models become more capable, they can generate audio-visual scenes exhibiting three forms of complexity: multi-event composition, multi-layer audio mixture, and asymmetric cross-modal relations. However, current evaluations largely reuse metrics from earlier simple and unidirectional tasks. They typically evaluate an audio-video clip as a whole and rely on simple predefined relations, making them unsuitable for these increasingly complex scenes. We propose an event-centric evaluation framework that disentangles audio and visual events at the event level, analyzes each event independently, cross-validates relations across modalities, and aggregates the judgments into a coarse-to-fine, three-level consistency metric. The metric assesses global narrative harmony, event-level temporal synchronization, and fine-grained attribute alignment, providing a general formulation for audio-video consistency evaluation across diverse generation settings and cross-modal relations. We instantiate the metric as AVC-Reward, a trainable audio-video reward model. To evaluate the agreement between automatic judgments and human perception across generation paradigms and relation complexities, we construct AVC-Benchmark, which is primarily built from diverse positive and negative samples generated by various types of audio-video generation models. AVC-Reward achieves state-of-the-art alignment with human judgment among automatic methods, provides interpretable rationales that localize failures, and improves the audio-video consistency of a joint generation model when used as an online reinforcement-learning signal. Project page: https://tsuki0512.github.io/AVC-Reward.github.io/
View arXiv page View PDF Project page GitHub Add to collection
Get this paper in your agent:
hf papers read 2610.22321
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 2
Text-to-Video • Updated 3 days ago • 2Tsuki0512/LTX-2-AVCReward-LoRA
Other • 32B • Updated 3 days ago • 40 • 1Tsuki0512/AVC-Reward
Datasets citing this paper 1
Tsuki0512/AVC-Benchmark
Viewer • Updated 3 days ago • 706 • 97 • 1
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2610.22321 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co