GPT-2 六個月後續:OpenAI 發佈 774 million 參數語言模型
GPT-2: 6-month follow-up
OpenAI 發佈 774 million 參數的 GPT-2 語言模型,接續 2 月推出的 124M 小型版及 5 月分階段發佈的 355M 中型版。OpenAI 與合作夥伴及 AI 社羣研究模型可能遭濫用及帶來社會效益。OpenAI 亦發佈開源法律協議,方便組織之間建立模型共享合作,並公佈一份關於與更廣泛 AI 研究社羣協調發表規範經驗的技術報告。
從 124M、355M 到 774 million 參數模型的分階段發佈,連同模型共享協議和發表規範協調經驗,呈現 GPT-2 後續開放與社羣協作的脈絡。
Research from these partners will factor into our future release decisions, as will observing how the 774M model is used, and discussing language models with researchers and policymakers to understand the considerations around larger models. As part of our staged release strategy, our current plan is to release the 1558M parameter model in a few months, but it’s plausible that findings from a partner, or malicious usage of our 774M model, could change this.
We think that a combination of staged release and partnership-based model sharing is likely to be a key foundation of responsible publication in AI, particularly in the context of powerful generative models. The issues inherent to large models are going to grow, rather than diminish, over time. We hope that our work on GPT‑2, discussed further in the technical report(opens in a new window) we’re publishing, will help provide evidence the AI community can draw on when thinking about the publication challenges inherent to some parts of AI research.
來源:OpenAI News · openai.com