Runway Gen-2：以文字、圖像或影片片段生成新影片
Gen-2: Generate novel videos with text, images or video clips
Runway 的 Gen-2 是一套多模態 AI 系統，可透過文字、圖像或影片片段生成新影片。頁面展示 8 種模式，包括文字轉影片、文字加圖像轉影片、圖像轉影片、風格轉換、Storyboard、Mask、Render 及模型自訂。用戶研究中，Gen-2 相較 Stable Diffusion 1.5 的偏好比例為 73.53%，相較 Text2Live 為 88.24%。
頁面把 Gen-2 的多種輸入與影片生成模式逐一列出，並附上兩項用戶偏好比例，能讓讀者掌握其功能範圍及比較結果。
Gen-2 is a multimodal AI system that can generate novel videos with text, images or video clips.
Anastasis Germanidis Feb 2023
No lights. No camera. All action.
Realistically and consistently synthesize new videos. Either by applying the composition and style of an image or text prompt to the structure of a source video (Video to Video). Or, using nothing but words (Text to Video). It's like filming something new, without filming anything at all.
Driving prompt: Top-down-drone-shot of icebergs with muted colors
Bringing the magic back to making movies.
Learn more about the different ways Gen-2 can turn any image, video clip or text prompt into a compelling piece of film.
Mode 01: Text to Video
Synthesize videos in any style you can imagine using nothing but a text prompt. If you can say it, now you can see it.
Driving prompt: The late afternoon sun peeking through the window of a New York City loft.
Mode 02: Text + Image to Video
Generate a video using a driving image and a text prompt
Input Image
Driving prompt: A low angle shot of a man walking down a street, illuminated by the neon signs of the bars around him
Mode 03: Image to Video
Generate video using just a driving image (Variations Mode)
Input Image
Output Video
Mode 04: Stylization
Transfer the style of any image or prompt to every frame of your video.
Input Image
Output Video
Mode 05: Storyboard
Turn mockups into fully stylized and animated renders.
Input Video
Generated Video
Mode 06: Mask
Apply stylization selectively to specific regions of your video while leaving the rest unchanged.
Input Video
Generated Video
Mode 07: Render
Turn untextured renders into realistic outputs by applying an input image or prompt.
Input Video
Generated Video
Mode 08: Customization
Unleash the full power of Gen-2 by customizing the model for even higher fidelity results.
Input Video
Generated Video
The New Standard for Video Generation
Based on user studies, results from Gen-2 are preferred over existing methods for image-to-image and video-to-video translation.
73.53%
Preferred over Stable Diffusion 1.5
88.24%
Preferred over Text2Live
A New Era for Motion (and) Pictures
Runway Research is dedicated to building the multimodal AI systems that will enable new forms of creativity. Gen-2 represents yet another of our pivotal steps forward in this mission.
來源：Runway：News（網頁） · runway.com