- Ruofei Du
- Na Li
- Jing Jin
- Michelle Carney
- Scott Joseph Miles
- Maria Kleiner
- Xiuxiu Yuan
- Yinda Zhang
- Anuva Kulkarni
- Xingyu “Bruce” Liu
- Ahmed K Sabie
- Sergio Orts Escolano
- Abhishek Kar
- Ping Yu
- Ram Iyengar
- Adarsh Kowdle
- Alex Olwal
Abstract
In recent years, there has been a proliferation of multimedia applications that leverage machine learning (ML) for interactive experiences. Prototyping ML-based applications is, however, still challenging, given complex workflows that are not ideal for design and experimentation. To better understand these challenges, we conducted a formative study with seven ML practitioners to gather insights about common ML evaluation workflows.
This study helped us derive six design goals, which informed Rapsai, a visual programming platform for rapid and iterative development of end-to-end ML-based multimedia applications. Rapsai is based on a node-graph editor to facilitate interactive characterization and visualization of ML model performance. Rapsai streamlines end-to-end prototyping with interactive data augmentation and model comparison capabilities in its no-coding environment. Our evaluation of Rapsai in four real-world case studies (N=15) suggests that practitioners can accelerate their workflow, make more informed decisions, analyze strengths and weaknesses, and holistically evaluate model behavior with real-world input.
Research Areas
Learn more about how we do research
We maintain a portfolio of research projects, providing individuals and teams the freedom to emphasize specific types of work