Kubeflow Overview
An open-source project that packages machine learning code for running on Kubernetes
Kubeflow Pipelines
A platform for composing, deploying and managing end-to-end ML workflows - promoting reusability and portability
Concepts
Reusability
Portability
ML Engineer can create a pipeline component/step for feature engineering, modelling and hyper-parameter tuning, while the Data Engineer can use that step as part of their pipeline as part of their data engineering solution. The solution can then appear as a service used by a Data Analyst to derive business insights.
Features
UI for managing and tracking experiments, jobs and runs, and sharing pipelines
Engine for scheduling multi-step ML workflows
SDK for defining and manipulating pipelines and components
Notebooks for interacting with the system using the SDK