Opportunities and Challenges Of Machine Learning Accelerators In Production

The rise of deep learning has resulted in tremendous demand for compute power, with the FLOPS required for leading machine learning (ML) research doubling roughly every 3.5 months since 2012 [1]. This increase in demand for compute has coincided with the end of Moore’s Law [2]. As a result, major industry players such as NVIDIA, Intel, and Google have invested in ML accelerators that are purpose built for deep learning workloads. ML accelerators present many opportunities and challenges in production environments. Workloads span a diverse portfolio including but not limited to visual search, fast text transcoding and translations, recommendations, and scientific computing such as climate prediction. This paper discusses some high level observations from experience internally at Google.

• Distributed Systems and Parallel Computing

• Machine Intelligence