PreView:透過預處理視圖與線索提升 GUI 定位
PreView: Enhancing GUI Grounding with Preprocessed Views and Cues
PreView 透過輔助預處理器改善 GUI 定位,為定位模型準備任務相關的裁剪及放大視圖,以及候選目標的視覺和文字線索。輕量輔助模型負責準備輸入,定位模型則保持凍結;PreView 在五個基準和三個定位模型上接受評估。
Hi everyone, author here! 👋
Can we improve GUI grounding by better preparing what the grounding model sees?
We introduce PreView, an auxiliary preprocessor that combines task-relevant cropped and enlarged views with visual and textual cues about candidate targets. The idea is simple: zooming helps reveal small controls, while candidate-specific cues help disambiguate which control the instruction refers to.
A trained lightweight auxiliary model prepares these inputs, while the grounding model stays frozen. We evaluate PreView across five benchmarks and three grounding models.
Happy to discuss the approach and hear your feedback!
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co