Deep learning in 3D with Facebook AI’s new tool PyTorch3D

Facebook AI is having a busy week—after the data visualization tool HiPlot, this week’s second deep learning tool PyTorch3D has been released. Developed by the Facebook AI Research Computer Vision Team a while back, it is now available open source on GitHub.

SEE ALSO: PyTorch 1.4 adds experimental Java bindings and additional PyTorch Mobile support

In PyTorch3D, Facebook AI sees “the potential for building systems that make high-quality 3D predictions without relying on time-intensive, manual 3D annotations,” as stated in the announcement blog post.

Let’s see what features the deep learning tool for 3D shape prediction offers.

Features

PyTorch3D is designed to integrate with deep learning methods for 3D data prediction and manipulation. For example, its operators are implemented using PyTorch tensors and it can utilize GPUs.

It comes with a differentiable mesh renderer and allows you to store and manipulate data meshes. Different operations can be performed on the meshes, namely sampling, loss functions, projective transformations and graph convolution.

The concept of triangle meshes is visualized in a video on the Facebook AI blog:

Source: video by Facebook AI

On GitHub, you can find tutorials for four different PyTorch3D use cases, ranging from deforming a sphere mesh into a dolphin to rendering textured meshes:

Source.

Mesh R-CNN

Mesh R-CNN, announced on the Facebook AI blog last October, is a method for predicting 3D shapes that was built with the help of PyTorch3D. Along with the open sourcing of PyTorch3D, Mesh R-CNN’s code is now available on GitHub as well.

SEE ALSO: OpenAI sets PyTorch as its new standard deep learning framework

Read more about PyTorch3D in the GitHub repo and in the blog post.

The post Deep learning in 3D with Facebook AI’s new tool PyTorch3D appeared first on JAXenter.