跳到正文
Runway:News(網頁)·· 2026-05-08AI 評分50

Runway 如何在生成式 AI 產品與流程中保障兒童安全

Our Approach to Child Safety

AI 導讀

Runway 在模型開發、產品部署及持續監察各階段設置措施,降低生成和散播涉及兒童的性剝削內容的風險。措施包括訓練數據篩查、模型測試、內容檢測、違規帳戶處置,以及向 NCMEC 舉報已確認的 CSAM;2025 年共提交 516 宗報告。Runway 亦採用 C2PA 來源訊號,並與 Thorn、Tech Coalition 等機構合作。

正文

Generative AI is changing what's possible in creative tools, and with that comes a responsibility to ensure those tools can't be used against the people most vulnerable to harm. Protecting children from sexual exploitation is one of our deepest held commitments as a company.

In line with our general approach to safety, we integrate child safety considerations at every level, from model development to product launches to end user generations. This approach closely aligns with Thorn's Safety by Design for Generative AI principles, which set an industry standard for how generative AI developers can guard against the creation and spread of child sexual abuse material (CSAM), including AI-generated CSAM, and other sexual harms against children. Below is a summary of how those principles show up across our products and processes.

1. Develop: Building Models That Proactively Address Risk

Safeguarding Training Data

Safety starts well before a user ever touches our product. We take deliberate steps during model development to reduce the risk that our models can be used to generate CSAM or other sexual content involving minors. We integrate hash matching, child safety classifiers and LLM-based moderation to ensure we do not train our models on sexual content involving minors or adults.

Red Teaming and Evaluation

Before a model ships, we conduct thorough testing to the extent legally permissible to identify and resolve potential vulnerabilities. We conduct such testing across text, image, video and audio to mitigate the possibility that CSAM or other sexual content involving minors could be produced by a user. This continuous testing ensures that our mitigations keep pace as models and new techniques emerge, and threat vectors evolve.

2. Deploy: Safeguards, Policies and Enforcement

Clearly Defining Usage Restrictions

We ensure that our strict boundary against any sexual content involving children is clear to all of our users. Our Usage Policy prohibits all "content that depicts, facilitates or promotes child sexual abuse or the sexualization of children," and clearly outlines that any violation will result in a permanent account ban, and, where appropriate, reporting to the National Center for Missing and Exploited Children (NCMEC).

Detecting CSAM & Sexual Content Involving Children

Once a model is deployed, we rely on multiple layers of detection to catch potentially harmful content, and attempts to create such content. This includes scanning all user-provided content against known-CSAM hash databases and a CSAM-specific classifier to detect previously unknown CSAM. We also apply AI-based classifiers to identify attempts to create CSAM or other sexual content involving children.

Reporting to NCMEC

We manually review all flagged content, and report all confirmed CSAM content to NCMEC. In 2025, we submitted a total of 516 reports to NCMEC's CyberTipline.

Deploying Content Provenance

We implement C2PA provenance signals so that content generated with our tools can be traced back to its origin. Provenance isn't a complete solution to misuse, but it gives platforms, researchers and law enforcement a meaningful signal for identifying AI-generated content.

3. Maintain: Monitoring, Improvement and Collaboration

Monitoring and Continuous Iteration

The techniques used to create and distribute CSAM evolve quickly. We are continuously testing and iterating on our models and safeguards to ensure that our safeguards can keep pace, especially as we move toward more content generated in real time.

Collaborating with Industry and Civil Society

We also recognize that no single company can solve this problem alone. We are committed to engaging with organizations like Thorn and Tech Coalition and with peers across the industry because the best defenses come from shared knowledge. We'll keep investing in these partnerships, and we'll keep updating our approach as the threat landscape changes and the tools to counter it evolve.

You can read more about our broader safety work on our Safety page. If you encounter content that you believe was created with our tools and raises concerns, please report it here.

來源:Runway:News(網頁) · runway.com