Lyft announced in a Medium blog post that they are making Flyte publicly available. Flyte is a structured programming and distributed processing platform for developing, executing and monitoring scalable machine learning workflows.
SEE ALSO: Meet Manifold: Uber’s machine learning model debugging tool goes open source
Built on Kubernetes
Flyte is built on Kubernetes to provide benefits such as portability, scalability and reliability. The platform consists of tasks that can be linked together in the form of directed acyclic graphs. It is built to scale and Lyft has been using it in production over various teams.
The following graphic shows how Flyte is designed:
Flyte’s architecture. Source.
Applying Machine Learning online at Scale
Jon Bratseth (Verizon)
Predictive Maintenance – how does Data Science revolutionize the World of Machines?
Victoriya Kalmanovich (Navy)
Scaling Machine Learning Systems up to Billions of Predictions per Day
Carmine Paolino (OLX Group)
Developing, executing and monitoring ML workflows
A graph is needed for creating a workflow, and Flyte offers developer tools to help ensure it meets the specifications. The documentation claims that, in only 20 lines of code, “a user can define a custom workflow, submit the graph to Flyte, and snooze while Flyte fulfills the execution on a schedule, or on-demand.”
Workflow execution is carried out using docker containers. Flyte supports distributed workflows, automatic retries and custom notifications.
Monitoring takes place throughout the workflow’s lifecycle, and the current status can be tracked via Flyte web interface. The web interface lets you take a closer look at the workflow steps and receive detailed error messages.
As an example, Lyft showed this workflow by Tim Wang on the Rider Pricing team as displayed in the Flyte UI:
Pricing Workflow by Tim Wang in Flyte UI. Source.
SEE ALSO: What is Kubernetes and how does it relate to Docker?
Flyte is now available on GitHub, and the Python SDK Flytekit for authoring, testing, deploying and interacting with workflows and tasks is on board as well.
The official website and the Medium blog post offer further details.
The post Lyft open sources Flyte for developing machine learning workflows appeared first on JAXenter.
Source : JAXenter
