OpenAI sets PyTorch as its new standard deep learning framework

PyTorch is now set to be OpenAI’s standard deep learning framework, as the capped-profit research organization for artificial intelligence announced in a blog post.

SEE ALSO: OpenAI finally releases “dangerous” large-scale unsupervised language model GPT-2

In general, OpenAI’s framework standardization is meant to benefit the creation and sharing of models, but why was PyTorch chosen for this task and what does this imply for its future development?

Performance benefits and developer community

The open source framework PyTorch, developed by Facebook and first released in 2016, comes with a Python and a C++ interface. The latest version, PyTorch 1.4, landed earlier this month and introduced some new features including experimental Java bindings.

OpenAI sees PyTorch’s main advantage in that it can increase research productivity at scale on GPUs. This is illustrated by the organization’s example of cutting down iteration time in generative modeling from weeks to days.

And also, OpenAI is “excited to be joining a rapidly-growing developer community, including organizations like Facebook and Microsoft, in pushing scale and performance on GPUs.”

In addition to using PyTorch in its research projects, OpenAI is also planning to contribute to the open source framework in the future. It remains to be seen how the developer community will react to this, as OpenAI has received some backlash in the past after initially keeping its NLP model GPT-2 under lock and key, until it was released nine months after its first public announcement.

SEE ALSO: “BERT is a system than can be tuned to do practically all tasks in NLP”

The transition from a non-profit to a “capped-profit” organization in March 2019 also led to some controversy on Twitter:

We’ve created OpenAI LP, a new “capped-profit” company that allows us to rapidly increase our investments in compute and talent while including checks and balances to actualize our mission. https://t.co/OAkatL2vyy pic.twitter.com/loqckUZPzB

— OpenAI (@OpenAI) March 11, 2019

Although PyTorch is now OpenAI’s standard framework, the organization clarified that exceptions are still possible when other frameworks are better suited for a certain project.

The OpenAI blog provides further details on the switch to PyTorch.

The post OpenAI sets PyTorch as its new standard deep learning framework appeared first on JAXenter.