Boris Smus
Authored Publications
Google Publications
Other Publications
Sort By
Modeling and Improving Text Stability in Live Captions
Xingyu "Bruce" Liu
Jun Zhang
Leonardo Ferrer
Susan Xu
Vikas Bahirwani
Extended Abstract of the 2023 CHI Conference on Human Factors in Computing Systems (CHI), ACM, 208:1-9
Preview abstract
In recent years, live captions have gained significant popularity through its availability in remote video conferences, mobile applications, and the web. Unlike preprocessed subtitles, live captions require real-time responsiveness by showing interim speech-to-text results. As the prediction confidence changes, the captions may update, leading to visual instability that interferes with the user’s viewing experience. In this work, we characterize the stability of live captions by proposing a vision-based flickering metric using luminance contrast and Discrete Fourier Transform. Additionally, we assess the effect of unstable captions on the viewer through task load index surveys. Our analysis reveals significant correlations between the viewer's experience and our proposed quantitative metric. To enhance the stability of live captions without compromising responsiveness, we propose the use of tokenized alignment, word updates with semantic similarity, and smooth animation. Results from a crowdsourced study (N=123), comparing four strategies, indicate that our stabilization algorithms lead to a significant reduction in viewer distraction and fatigue, while increasing viewers' reading comfort.
View details
Unmet Needs and Opportunities for Mobile Translation AI
Abigail Evans
Aaron Michael Donsbach
Jess Scon Holbrook
Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI ’20), ACM, Honolulu, Hawaii, USA
Preview abstract
Translation apps and devices are often presented in the context of providing assistance while traveling abroad. However, the spectrum of needs for cross-language communication is much wider. To investigate these needs, we conducted three studies with populations spanning socioeconomic status and geographic regions: (1) United States-based travelers, (2) migrant workers in India, and (3) immigrant populations in the United States. We compare frequent travelers' perception and actual translation needs with those of the two migrant communities. The latter two, with low language proficiency, have the greatest translation needs to navigate their daily lives. However, current mobile translation apps do not meet these needs. Our findings provide new insights on the usage practices and limitations of mobile translation tools. Finally, we propose design implications to help apps better serve these unmet needs.
View details
Preview abstract
Micro-task markets such as Amazon’s Mechanical Turk represent a new paradigm for accomplishing work, in which employers can tap into a large population of workers around the globe to accomplish tasks in a fraction of the time and money of more traditional methods. However, such markets have been primarily used for simple, independent tasks, such as labeling an image or judging the relevance of a search result. Here we present a general purpose framework for accomplishing complex and interdependent tasks using micro-task markets. We describe our framework, a web-based prototype, and case studies on article writing, decision making, and science journalism that demonstrate the benefits and limitations of the approach.
View details
No Results Found
