EmbeddingGemma 2: An open, lightweight multimodal embedding model
EmbeddingGemma 2: An open, lightweight multimodal embedding model
EmbeddingGemma 2 是一款開放、輕量的多模態嵌入模型,討論提及視覺、影片及音訊應用,並提出以此建立本地多模態搜尋的可能性。評論者觀察到,圖像及音訊嵌入可同時承載文字內容與風格、視聽語調資訊,兩者似乎可以線性分離。
Not just vision with video, but also audio, it really seems amazing.
I’m not sure how it can handle vicinity of pairs of embeddings with for example some words and the audio where they’re spoken or an image where the text is handled. Building local multimodal search with this would be amazing. I’ve explored this stuff with CLIP and it’s interesting how image (but also audio) embedding carries both the clean “text” content information but also the stylistic and visual/audio tone information, the two can even kind of be linearly separated.
來源:Hacker News:AI 熱帖 · news.ycombinator.com