Screenplay 開源工具讓編碼智能體製作螢幕錄影

I like Screen Studio-style videos: a smooth cursor, zooms that follow the action, a nice gradient behind the window. Making them by hand means recording, re-recording, then editing. So I built Screenplay, an open-source tool that lets your coding agent do it.

Repo (MIT): https://github.com/gmpetrov/agent-screenplay

How it works

You ask your agent (Claude Code, Codex, Cursor or any MCP client) for a video:

/screenplay Make a 30-second walkthrough of https://yoursite.com

1. The agent opens the site, explores it, and writes a short JSON scenario: open this page, click this button, type that, hold here, zoom there.
2. Screenplay runs check, which plays the steps fast without filming, so broken selectors show up before the real take.
3. record plays the scenario in headless Chromium (Playwright), films it, then edits it: smoothed cursor, automatic zooms, click ripples, callouts, spotlight, captions and an optional voice-over.
4. The agent looks at the storyboard image of the take and fixes the scenario if a shot is off. Re-editing the look doesn't need a new take: render.

Desktop apps work too (macOS, Windows, Linux): the same kind of scenario is played through the OS accessibility APIs, and the window is filmed with ffmpeg.

A scenario

Targets are named the way people see them (text=…, role=button[name='…'], label=…), so the agent doesn't need CSS selectors:

{
  "url": "https://app.example.com/inbox",
  "style": { "background": "dawn" },
  "steps": [
    { "click": "text=Camille Laurent", "say": "When the AI agent gets stuck, the conversation lands in your inbox." },
    { "callout": "role=button[name='Take over']", "text": "Take over from the AI agent" },
    { "click": "role=button[name='Take over']", "say": "One click, and the conversation is yours." },
    { "type": "Hi Camille, I'm on it.", "into": "placeholder=Write a reply…" },
    { "click": "role=button[name='Send']" }
  ]
}

Design choices

• No AI inside, no API key. Your agent does the thinking; Screenplay only plays, films and edits. It's a CLI plus an MCP server.
• Headless first. Coding agents often run in containers without a screen, and they still need to film.
• Own renderer. Frames are drawn with @napi-rs/canvas and encoded with ffmpeg (both ship with the package). I didn't want a rendering library whose licence makes companies pay, in an MIT tool.
• Voice-over with Cartesia or ElevenLabs, or with your own audio files (from any TTS, local models included). The video waits for each sentence, so the voice sets the pace.

Try it

npx agent-screenplay init

init downloads Chromium once, adds the MCP server to Claude Code, Codex and Cursor, and installs the guide that teaches the agent to write good scenarios. Restart your agent, then ask for a video.

It's brand new, so I'd love feedback, especially on the scenario format and the default look. Issues and stars are very welcome: https://github.com/gmpetrov/agent-screenplay

Written with an AI agent and published at my request; every fact comes from the repo.