Narayan Hegde
Working on understanding and developing deep neural networks for healthcare applications
Authored Publications
Sort By
Deep learning models for histologic grading of breast cancer and association with disease prognosis
Trissia Brown
Isabelle Flament
Fraser Tan
Yuannan Cai
Kunal Nagpal
Emad Rakha
David J. Dabbs
Niels Olson
James H. Wren
Elaine E. Thompson
Erik Seetao
Carrie Robinson
Melissa Miao
Fabien Beckers
Lily Hao Yi Peng
Craig Mermel
Cameron Chen
npj Breast Cancer (2022)
Preview abstract
Histologic grading of breast cancer involves review and scoring of three well-established morphologic features: mitotic count, nuclear pleomorphism, and tubule formation. Taken together, these features form the basis of the Nottingham Grading System which is used to inform breast cancer characterization and prognosis. In this study, we developed deep learning models to perform histologic scoring of all three components using digitized hematoxylin and eosin-stained slides containing invasive breast carcinoma. We then evaluated the prognostic potential of these models using an external test set and progression free interval as the primary outcome. The individual component models performed at or above published benchmarks for algorithm-based grading approaches and achieved high concordance rates in comparison to pathologist grading. Prognostic performance of histologic scoring provided by the deep learning-based grading was on par with that of pathologists performing review of matched slides. Additionally, by providing scores for each component feature, the deep-learning based approach provided the potential to identify the grading components contributing most to prognostic value. This may enable optimized prognostic models as well as opportunities to improve access to consistent grading and better understand the links between histologic features and clinical outcomes in breast cancer.
View details
Walking with PACE — Personalized and Automated Coaching Engine
Deepak Nathani
Eshan Motwani
Karina Lorenzana Livingston
Madhurima Vardhan
Martin Gamunu Seneviratne
Nur Muhammad
Rahul Singh
Shantanu Prabhat
Sriram Lakshminarasimhan
Srujana Merugu
UMAP: 30th ACM Conference on User Modeling, Adaptation and Personalization (2022)
Preview abstract
Fitness coaching is effective in helping individuals to develop and maintain healthy lifestyle habits. However, there is a significant shortage of fitness coaches, particularly in low resource communities. Automated coaching assistants may help to improve the accessibility of personalized fitness coaching. Although a variety of automated nudge systems have been developed, few make use of formal behavior science principles and they are limited in their level of personalization. In this work, we introduce a computational framework leveraging the Fogg’s behavioral science model which serves as a personalised and automated coaching engine (PACE).PACE is a rule-based system that infers user state and suggests appropriate text nudges. We compared the effectiveness of PACE to human coaches in a Wizard-of-Oz deployment study with 33 participants over 21 days. Participants were randomized to either a human coach (’human’ arm, n=18) or the PACE framework handled by a human coach (’wizard’ arm, n=15). Coaches and participants interacted via a chat interface. We tracked coach-participant conversations, step counts and qualitative survey feedback. Our findings indicate that the PACE framework strongly emulated human coaching with no significant differences in the overall number of active days (PACE: 85.33% vs human: 92%, [p=NS]) and step count (PACE: 6674 vs human: 6605, [p=NS]) of participants from both groups.The qualitative user feedback suggests that PACE cultivated a coach-like experience, offering barrier resolution, motivation and educational support. As a post-hoc analysis, we annotated the conversation logs from the human coaching arm based on the Fogg framework, and then trained machine learning (ML) models on these data sets to predict the next coach action (AUC 0.73±0.02). This suggests that a ML-driven approach may be a viable alternative to a rule-based system in suggesting personalized nudges. In future, such an ML system could be made increasingly personalized and adaptive based on user behaviors.
View details
Interpretable Survival Prediction for Colorectal Cancer using Deep Learning
Melissa Moran
Markus Plass
Robert Reihs
Fraser Tan
Isabelle Flament
Trissia Brown
Peter Regitnig
Cameron Chen
Apaar Sadhwani
Bob MacDonald
Benny Ayalew
Lily Hao Yi Peng
Heimo Mueller
Zhaoyang Xu
Martin Stumpe
Kurt Zatloukal
Craig Mermel
npj Digital Medicine (2021)
Preview abstract
Deriving interpretable prognostic features from deep-learning-based prognostic histopathology models remains a challenge. In this study, we developed a deep learning system (DLS) for predicting disease-specific survival for stage II and III colorectal cancer using 3652 cases (27,300 slides). When evaluated on two validation datasets containing 1239 cases (9340 slides) and 738 cases (7140 slides), respectively, the DLS achieved a 5-year disease-specific survival AUC of 0.70 (95% CI: 0.66–0.73) and 0.69 (95% CI: 0.64–0.72), and added significant predictive value to a set of nine clinicopathologic features. To interpret the DLS, we explored the ability of different human-interpretable features to explain the variance in DLS scores. We observed that clinicopathologic features such as T-category, N-category, and grade explained a small fraction of the variance in DLS scores (R2 = 18% in both validation sets). Next, we generated human-interpretable histologic features by clustering embeddings from a deep-learning-based image-similarity model and showed that they explained the majority of the variance (R2 of 73–80%). Furthermore, the clustering-derived feature most strongly associated with high DLS scores was also highly prognostic in isolation. With a distinct visual appearance (poorly differentiated tumor cell clusters adjacent to adipose tissue), this feature was identified by annotators with 87.0–95.5% accuracy. Our approach can be used to explain predictions from a prognostic deep learning model and uncover potentially-novel prognostic features that can be reliably identified by people for future validation studies.
View details
Similar Image Search for Histopathology: SMILY
Jason Hipp
Michael Emmert-Buck
Daniel Smilkov
Mahul Amin
Craig Mermel
Lily Peng
Martin Stumpe
Nature Partner Journal (npj) Digital Medicine (2019)
Preview abstract
The increasing availability of large institutional and public histopathology image datasets is enabling the searching of these datasets for diagnosis, research, and education. Although these datasets typically have associated metadata such as diagnosis or clinical notes, even carefully curated datasets rarely contain annotations of the location of regions of interest on each image. As pathology images are extremely large (up to 100,000 pixels in each dimension), further laborious visual search of each image may be needed to find the feature of interest. In this paper, we introduce a deep-learning-based reverse image search tool for histopathology images: Similar Medical Images Like Yours (SMILY). We assessed SMILY’s ability to retrieve search results in two ways: using pathologist-provided annotations, and via prospective studies where pathologists evaluated the quality of SMILY search results. As a negative control in the second evaluation, pathologists were blinded to whether search results were retrieved by SMILY or randomly. In both types of assessments, SMILY was able to retrieve search results with similar histologic features, organ site, and prostate cancer Gleason grade compared with the original query. SMILY may be a useful general purpose tool in the pathologist’s arsenal, to improve the efficiency of searching large archives of histopathology images, without the need to develop and implement specific tools for each application.
View details
Human-Centered Tools for Coping with Imperfect Algorithms during Medical Decision-Making
Jason Hipp
Daniel Smilkov
Martin Stumpe
Conference on Human Factors in Computing Systems (2019)
Preview abstract
Machine learning (ML) is increasingly being used in image retrieval systems for medical decision making. One application of ML is to retrieve visually similar medical images from past patients (e.g. tissue from biopsies) to reference when making a medical decision with a new patient. However, no algorithm can perfectly capture an expert's ideal notion of similarity for every case: an image that is algorithmically determined to be similar may not be medically relevant to a doctor's specific diagnostic needs. In this paper, we identified the needs of pathologists when searching for similar images retrieved using a deep learning algorithm, and developed tools that empower users to cope with the search algorithm on-the-fly, communicating what types of similarity are most important at different moments in time. In two evaluations with pathologists, we found that these refinement tools increased the diagnostic utility of images found and increased user trust in the algorithm. The tools were preferred over a traditional interface, without a loss in diagnostic accuracy. We also observed that users adopted new strategies when using refinement tools, re-purposing them to test and understand the underlying algorithm and to disambiguate ML errors from their own errors. Taken together, these findings inform future human-ML collaborative systems for expert decision-making.
View details
