Laurent El Shafey
Authored Publications
Sort By
Pathways: Asynchronous Distributed Dataflow for ML
Aakanksha Chowdhery
Ruoming Pang
Sudip Roy
Brennan Saeta
Parker Edward Schuh
Ryan Sepassi
MLSys 2022 (2022) (to appear)
Preview abstract
We present the design of a new large scale orchestration layer for accelerators. Our system, Pathways, is explicitly designed to enable exploration of new systems and ML research ideas, while retaining state of the art performance for current models. Pathways uses a sharded dataflow graph of asynchronous operators that consume and produce futures, and efficiently gang-schedules heterogeneous parallel computations on thousands of accelerators while coordinating data transfers over their dedicated interconnects. Pathways makes use of a novel asynchronous distributed dataflow design that lets the control plane execute in parallel despite dependencies in the data plane. This design, with careful engineering, allows Pathways to adopt a single-controller model that makes it easier to express complex new parallelism patterns. We demonstrate that Pathways can achieve performance parity (~100% accelerator utilization) with state-of-the-art systems when running SPMD computations over 2048 TPUs, while also delivering throughput comparable to the SPMD case for Transformer models that are pipelined across 16 stages, or sharded across two islands of accelerators connected over a data center network.
View details
The Medical Scribe: Corpus Development and Model Performance Analyses
Amanda Perry
Ashley Robson Domin
Chris Co
Gang Li
Hagen Soltau
Justin Stuart Paul
Lauren Keyes
Linh Tran
Mark David Knichel
Mingqiu Wang
Nan Du
Rayman Huang
Proc. Language Resources and Evaluation, 2020
Preview abstract
There has been a growing interest in creating tools to assist clinical note generation from the audio of provider-patient encounters. Motivated by this goal and with the help of providers and experienced medical scribes, we developed an annotation scheme to extract relevant clinical concepts. Using this annotation scheme, a corpus of about 6k clinical encounters was labeled, which was used to train a state-of-the-art tagging model. We report model performance and a detailed analyses of the results.
View details
Preview abstract
Speech applications dealing with conversations require not only recognizing the spoken words, but also determining who spoke when. The task of assigning words to speakers is typically addressed by merging the outputs of two separate systems, namely, an automatic speech recognition (ASR) system and a speaker diarization (SD) system. The two systems are trained independently with different objective functions. Often the SD systems operate directly on the acoustics and are not constrained to respect word boundaries and this deficiency is overcome in an ad hoc manner. Motivated by recent advances in sequence to sequence learning, we propose a novel approach to tackle the two tasks by a joint ASR and SD system using a recurrent neural network transducer. Our approach utilizes both linguistic and acoustic cues to infer speaker roles, as opposed to typical SD subsystems, which only use acoustic cues. We evaluate the performance of our model on a large corpus of medical conversations between physicians and patients and find that our approach improves performance by about 86% word-level diarization error rate over a competitive conventional baseline.
View details
