ItemSuggest: A Data Management Platform for Machine Learned Ranking Services

Machine Learning (ML) is a critical component of several novel applications and intelligent features in existing applications. Recent advances in deep learning have fundamentally advanced the state- of-the-art in several areas of research and made it easier to apply ML to a wide variety of problems. However, applied ML projects in industry, where the objective is to build and improve a production feature that uses ML continues to be complicated and often bottlenecked by data management challenges. In this paper, we describe the design and implementation of a machine learning platform for building learned ranking services that leverages key ideas from data management. The platform allows engineers to focus on application-specific modeling and simplifies key tasks of 1) gathering training data, 2) cleaning, validating, and monitoring data quality, 3) training and evaluating models, 4) feature lifecycle management, 5) and infrastructure for A/B tests. We describe key design choices anchored around the core idea of optimizing for experiment velocity. We describe lessons learned from applications built on this platform that have been in production serving hundreds of millions of users for over a year. Finally, we identify two key components of the platform where data management research can have a major impact. We believe such platforms have the potential to accelerate and simplify ML applications the same way data warehouses radically simplified complex reporting applications.

• Machine Intelligence

• Data Management