OpenAI Refuses to Release GPT-2, Fearing Malicious Use
Due to concerns about malicious applications, GPT2 will not be released (2019)
OpenAI trained GPT-2, a 1.5-billion-parameter language model, on 40GB of internet text. It achieves state-of-the-art results on many benchmarks and can perform reading comprehension, translation, and summarization without task-specific training. But because the model could be used to generate misleading news, impersonate people, or automate spam, OpenAI is not releasing the full model or dataset. Instead, they are publishing a smaller version and a technical paper, and will discuss the decision in six months.
Due to our concerns about malicious applications of the technology, we are not releasing the trained model.
- wmf
I still think this was a fire drill. They knew it wasn't dangerous but they wanted the public to do their homework and come to the conclusion themselves.
- zenoprax
The samples are striking in their simplicity compared to the over-controlled system prompts we have now. For example, the intended format is as follows:
"GPT‑2 generates synthetic text samples in response to the model being primed with an arbitrary input. The model is chameleon-like—it adapts to the style and content of the conditioning text."
The results don't sound anything like today's models but note that each one took 10 attempts:
> ## System Prompt (human-written)
> In a shocking finding, scientist discovered a herd of unicorns living in a remote, previously unexplored valley, in the Andes Mountains. Even more surprising to the researchers was the fact that the unicorns spoke perfect English.
> ## Model Completion (machine-written, 10 tries)
> The scientist named the population, after their distinctive horn, Ovid’s Unicorn. These four-horned, silver-white unicorns were previously unknown to science.
> Now, after almost two centuries, the mystery of what sparked this odd phenomenon is finally solved.
> Dr. Jorge Pérez, an evolutionary biologist [...]
- mudkipdev
I wonder what they saw. GPT-2 has no malware applications. Maybe they were concerned about someone using it for spam?
- andy99
See also the 2023 call to “ pause for at least 6 months the training of AI systems more powerful than GPT-4.” https://futureoflife.org/open-letter/pause-giant-ai-experime...
Signed by Benjio, Musk, Woz, etc
- uejfiweun
I'm starting to think that AI development might be the great filter. There are so many levels of competition at play here and even if an agreement is reached to slow everything down, the incentive is to cheat at that. I really don't think there's much of a chance we have control over the situation, we're just gonna summon these things and all we can do is pray they don't want to kill us.