People + AI Research (PAIR)

Human-centered research and design to make AI partnerships productive, enjoyable, and fair.

About the team

The past few years have seen rapid advances in machine learning, with new technologies achieving dramatic improvements in technical performance. But we can go beyond optimizing objective functions. By building AI systems with users in mind from the ground up, we open up entire new areas of design and interaction.

PAIR is devoted to advancing the research and design of people-centric AI systems. We're interested in the full spectrum of human interaction with machine intelligence, from supporting engineers to understanding everyday experiences with AI.

Our goal is to do fundamental research, invent new technology, and create frameworks for design in order to drive a human-centered approach to artificial intelligence. And we want to be as open as possible: we’re building open source tools that everyone can use, hosting public events, and supporting academics in advancing the state of the art.

Our work

People + AI Guidebook

Model interpretibliity

Can machines explain the "why" behind their decisions? We're investigating ways for people to understand more about ML models, starting with visualizations that look under the hood of complex systems. See this video: Visualizing High-Dimensional Space.

Tools for health

We are investigating how AI technology and design can help the practice of clinicians and medical researchers

Design & Machine Learning

How ML is changing the way we build experiences and interact with the world.

PAIR Writer in Residence blog

Read the "AI Outside In" blog by our writer-in-residence, independent tech journalist and researcher David Weinberger. David explains central concepts in the field of machine learning, within a broader context of social issues and ideas.

Explaining fairness

Policy tradeoffs in machine learning can be complex, but visualizations and interactive explanations can help people understand these critical issues.

Forecasting earthquake aftershock locations with AI-assisted science

Machine learning-based forecasts may one day help deploy emergency services and inform evacuation plans for areas at risk of an aftershock.

See how the world draws

People who played the Quick, Draw! Game created a dataset that reflects a variety of perspectives around the globe. See visualizations of these 50 million drawings and a video about why diversity matters to machine learning.

Open-source software

What-If Tool: Analyze ML models with minimal code required

TensorFlow.js

TensorFlow.js is an open-source library for hardware-accelerated machine learning on the web. Train neural nets entirely in your browser, or run pre-trained models.

TCAV

We are designing machine learning algorithms to incorporate human knowledge and be more controllable and interpretable, without sacrificing accuracy.

UMAP.js

An open source, state-of-the-art dimensionality reduction algorithm, ported to JavaScript. Create interactive visualizations of high-dimensional data in the browser.

Embedding Projector

An open source tool that helps you to interactively visualize high-dimensional data. Read embeddings from your model and visualize them in two or three dimensions.

Facets

Training data can be seen as the raw ingredients for a machine learning system. A central focus of our group is helping engineers understand what goes into their models, as in the open-source Facets package.

GAN Playground

Explore GAN training dynamics with this interactive visualization.

TensorFlow Playground

Education is central to our mission. The Playground is an open-source neural net you can play with in your browser, allowing users to experiment with different architectures through a graphical UI.

Featured publications

Embedding Projector: Interactive Visualization and Interpretation of Embeddings

Daniel Smilkov, Nikhil Thorat, Charles Nicholson, Emily Reif, Fernanda B. Viégas, Martin Wattenberg

Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation

We propose a simple, elegant solution to use a single Neural Machine Translation (NMT) model to translate between multiple languages. Our solution requires no change in the model architecture from our base system but instead introduces an artificial token at the beginning of the input sentence to specify the required target language. The rest of the model, which includes encoder, decoder and...

View details

Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation

Melvin Johnson, Mike Schuster, Quoc V. Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenberg, Greg Corrado, Macduff Hughes, Jeffrey Dean

Google (2016)

Direct-Manipulation Visualization of Deep Networks

Daniel Smilkov, Shan Carter, D. Sculley, Fernanda B. Viégas, Martin Wattenberg

How to Use t-SNE Effectively

Martin Wattenberg, Fernanda B. Viégas, Ian Johnson

Design and Redesign in Data Visualization

Fernanda B. Viégas, Martin Wattenberg

Visualizing Statistical Mix Effects and Simpson's Paradox

We discuss how “mix effects” can surprise users of visualizations and potentially lead them to incorrect conclusions. This statistical issue (also known as “omitted variable bias” or, in extreme cases, as “Simpson’s paradox”) is widespread and can affect any visualization in which the quantity of interest is an aggregated value such as a weighted sum or average. Our first contribution is to...

View details

Visualizing Statistical Mix Effects and Simpson's Paradox

Zan Armstrong, Martin Wattenberg

Proceedings of IEEE InfoVis 2014, IEEE (to appear)

Ad Click Prediction: a View from the Trenches

Predicting ad click--through rates (CTR) is a massive-scale learning problem that is central to the multi-billion dollar online advertising industry. We present a selection of case studies and topics drawn from recent experiments in the setting of a deployed CTR prediction system. These include improvements in the context of traditional supervised learning based on an FTRL-Proximal online...

View details

Ad Click Prediction: a View from the Trenches

H. Brendan McMahan, Gary Holt, D. Sculley, Michael Young, Dietmar Ebner, Julian Grady, Lan Nie, Todd Phillips, Eugene Davydov, Daniel Golovin, Sharat Chikkerur, Dan Liu, Martin Wattenberg, Arnar Mar Hrafnkelsson, Tom Boulos, Jeremy Kubica

Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) (2013)

Interactive Visualization of Spatially Amplified GNSS Time-Series Position Fields

Brendan J. Meade, William T. Freeman, James Wilson, Fernanda B. Viégas, Martin Wattenberg

Visualizing Dataflow Graphs of Deep Learning Models in TensorFlow

We present a design study of the TensorFlow Graph Visualizer, part of the TensorFlow machine intelligence platform. This tool helps users understand complex machine learning architectures by visualizing their underlying dataflow graphs. The tool works by applying a series of graph transformations that enable standard layout techniques to produce a legible interactive diagram. To declutter the...

View details

Visualizing Dataflow Graphs of Deep Learning Models in TensorFlow

Kanit Wongsuphasawat, Daniel Smilkov, James Wexler, Jimbo Wilson, Dandelion Mane, Doug Fritz, Dilip Krishnan, Fernanda Viegas, Martin Wattenberg

IEEE Transaction on Visualization and Computer Graphics (2017)

SmoothGrad: removing noise by adding noise

Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, Martin Wattenberg

What Is Better Than Coulomb Failure Stress? A Ranking of Scalar Static Stress Triggering Mechanisms from 105 Mainshock-Aftershock Pairs

Brendan J. Meade, Phoebe M. R. DeVries, Jeremy Faller, Fernanda Viegas, Martin Wattenberg

Some of our people

Advancing the state of the art in machine learning means thinking beyond narrowly-defined objective functions. It requires weaving user needs and societal values into the design and evaluation of these systems.

Fernanda Viegas Senior Staff Research Scientist

Jess Holbrook

Martin Wattenberg

Been Kim

Carrie Jun Cai

Emily Reif

James Wexler

Tolga Bolukbasi

Mahima Pushkarna

Daniel Smilkov

Nithya Sambasivan

Adam Pearce

Jimbo Wilson

Courtney Heldreth

Lucas Dixon

Join us

We're looking for talented people to join our team

See opportunities