跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 11 天前AI 評分49

Open Vision Banana:生成式視覺感知的全開放方案

Open Vision Banana: A Fully Open Recipe for Generative Visual Perception

AI 導讀

Open Vision Banana(OVB)提出一套全開放、可重現的方案,把預訓練生成模型轉化為統一的多任務感知系統。OVB 使用 4B 和 9B 參數的開放權重基礎模型及 2.1M-3.8M 個訓練配對,在 Cityscapes 零樣本表現以 73.9 mIoU 超越 Vision Banana 報告的 69.9 mIoU。

正文

Published on Nov 11

Authors:

,

,

,

,

,

Abstract

Recent work suggests that pretrained image generators encode real-world understanding beyond image synthesis. Vision Banana exploits this by framing vision tasks as conditional image editing. However, its data, weights, code, and implementation details are not public. We present Open Vision Banana (OVB), a fully open and reproducible recipe for converting pretrained generators into unified multi-task perception systems. Our analyses identify the design choices that matter most for generalization, including color encoding, prompt design, parameter adaptation, and task balancing. These choices interact, so individually helpful choices can hurt when combined with others. Starting from 4B and 9B parameter open-weight base models and using 2.1M-3.8M training pairs from public datasets, OVB surpasses Vision Banana's reported Cityscapes zero-shot performance (73.9 vs. 69.9 mIoU) and rivals state-of-the-art affine-invariant depth estimators. Its non-zero-shot variants come within five points of Vision Banana on referring and reasoning segmentation but remain further behind on instance segmentation. All models largely retain the base model's image-editing abilities. With legend ensembling, we introduce a test-time scaling method for generator-based perception. It segments an image under several class-color legends and fuses the results by majority vote, correcting unstable color-class bindings and improving mIoU by up to 8.7 points on Cityscapes and 15.0 points on COCO-Object without retraining. We release the dataset construction and processing pipeline, the training and evaluation code, as well as the final, intermediate, and ablation checkpoints. Project website: https://openvisionbanana.github.io

View arXiv page View PDF Project page GitHub Add to collection

Get this paper in your agent:

hf papers read 2611.15844

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 1

FunAILab/Open-Vision-Banana

Image-to-Image •

Updated about 7 hours ago

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.15844 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.15844 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co