In this era of big data, sometimes it can be difficult to separate the signal from the noise. Thanks to a new platform open sourced by Facebook, all ML engineers need to do is head for the Horizon.
Horizon is an end-to-end platform for applied reinforcement learning built in Python. It utilizes PyTorch for modeling and training and Caffe2 for model serving. This open source platform includes workflows to train popular reinforcement learning algorithms, along with data preprocessing, feature transformation, distributed training, counterfactual policy evaluation, and optimized serving.
What is reinforcement learning?
If you’ve been following along with our “Know Your History” series, you’ll remember that artificial intelligence has a number of interesting offshoots, including machine learning. Reinforcement learning (RL for short) is the latest attempt to make our algorithms better.
Reinforcement learning’s main focus is on performance. In general, reinforcement learning is an approach based on how software agents should take actions in an environment so as to maximize some notion of cumulative reward.
As the Facebook devs put it, machine learning systems require oversight from engineers to put ML predictions into practice. With reinforcement learning, the system itself can make decisions, take actions, and then adapt those decisions based on feedback.
However, bringing this kind of optimized performance into production has been somewhat difficult. That’s where Facebook’s Horizon comes in.
SEE ALSO: AI-powered bug spray: Facebook’s new tool SapFix debugs code with AI
Horizon’s broad view
Horizon was designed to bridge the gap between the lab and the real world. In particular, Horizon pays close attention to issues that are specific to production environments, making sure they are better supported. This includes features like feature normalization, distributed training, large-scale deployment and serving.
More importantly, Horizon was designed for large, complex data sets and scales wonderfully. Applied RL models are often more sensitive to noisy and unnormalized data than traditional deep networks. Because Horizon preprocesses the data with Apache Spark, developers can use PyTorch based algorithms for normalization and training.
Horizon’s pipeline process. Source: Facebook.
It doesn’t matter how much computing power you have available, either. Horizon works equally well on a single GPU or CPU as it does on large clusters. That said, it is actually optimized for use on large clusters, as distributed training allows developers to solve for more complex problems and iterate the models even faster.
SEE ALSO: Keep lawsuits out of open source with GPLv3 extension, says Red Hat, Google, & Facebook
Aim for the Horizon
Want to try out Horizon for yourself? This end-to-end platform for scale RL work is available for developers on GitHub now! It can be installed via Docker or manually.
If you’d like to learn more about Horizon, Facebook has released a whitepaper, which is available for free here.
The post The first of its kind: Harness reinforcement learning in production with Facebook’s Horizon appeared first on JAXenter.
Source : JAXenter
