Common problems with Creating Machine Learning Pipelines from Existing Code
Abstract
We worked with over 100 participants in industry on developing machine learning (ML) pipelines. Working alongside ML platform owners, software engineers, devops engineers, and data scientists across industries, we migrated existing ML projects into ones with ML pipelines software systems, Kubeflow Pipelines (KFP) and Tensorflow Extended (TFX). In this workshop paper, we share common problems we observed when migrating existing ML code to an ML pipeline system.