Connectomics

Our goal is to leverage Google expertise and resources to advance understanding of the structure and function of the brain.

About our team

A major hypothesis in modern neuroscience is that neuron-to-neuron connectivity structure in the brain can be linked to function -- how the brain encodes memories, extracts features from perceptual stimuli, and makes decisions. However, the structure of these brain networks has remained largely unknown, due to technical difficulties involved in imaging and reconstructing the brain in 3D.

New microscopy techniques have begun to address the challenge of imaging the brain in 3D at nanometer resolution, and 4D at cellular resolution, but this has led to a huge bottleneck in the subsequent step of data analysis. Our goal is to help solve some of these data analysis problems and thus enable a high-throughput approach to studying the network architecture of the brain.

In order to facilitate this work we collaborate with the Max Planck Institute, HHMI Janelia Research Campus, Harvard University, and other organizations.

Research areas

Health & bioscience

General science

Machine perception

Team focus summaries

Automated 3d brain reconstruction

We develop algorithms and software for automating the process of aligning, segmenting, and annotating petabyte-scale 3d images of brain tissue.

Visualization and infrastructure

We develop software such as TensorStore and Neuroglancer which helps store, process, and visualize large n-dimensional images and volumes.

Highlighted projects

A Browsable Petascale Reconstruction of the Human Cortex

In collaboration with the Lichtman Laboratory at Harvard University, we are releasing the “H01” dataset, a 1.4 petabyte rendering of a small sample of human brain tissue, along with a companion paper, “A connectomic study of a petascale fragment of human cerebral cortex.

The Drosophila Hemibrain Connectome: The Largest Synapse-Resolution Map of Brain Connectivity

In collaboration with the FlyEM team at HHMI’s Janelia Research Campus and several other research partners, we released the "hemibrain" connectome, a highly detailed map of neuronal connectivity in the fly brain, along with a suite of tools for visualization and analysis.

Flood-Filling Networks: Improving Connectomics by an Order of Magnitude

In collaboration with colleagues at the Max Planck Institute of Neurobiology, we published "High-Precision Automated Reconstruction of Neurons with Flood-Filling Networks" in Nature Methods, which shows how a new type of recurrent neural network can improve the accuracy of automated interpretation of connectomics data by an order of magnitude over previous deep learning techniques.

Neuroglancer

Neuroglancer is a WebGL-based viewer for petascale multidimensional data. It is capable of displaying arbitrary (non axis-aligned) cross-sectional views of volumetric data, as well as 3-D meshes and line-segment based models (skeletons).

TensorStore

Cloud-first library for reading and writing large multi-dimensional arrays, with advanced concurrency and indexing capabilities.

Featured publications

A connectomic study of a petascale fragment of human cerebral cortex

We acquired a rapidly preserved human surgical sample from the temporal lobe of the cerebral cortex. We stained a 1 mm3 volume with heavy metals, embedded it in resin, cut more than 5000 slices at ∼30 nm and imaged these sections using a high-speed multibeam scanning electron microscope. We used computational methods to render the three-dimensional structure containing 57,216 cells, hundreds of...

View details

A connectomic study of a petascale fragment of human cerebral cortex

Alex Shapson-Coe, Michal Januszewski, Daniel R. Berger, Art Pope, Yuelong Wu, Tim Blakely, Richard L. Schalek, Peter Hawley Li, Shuohong Wang, Jeremy Maitin-Shepard, Neha Karlupia, Sven Dorkenwald, Evelina Sjostedt, Laramie J Leavitt, Dongil Lee, Luke Bailey, Angerica Fitzmaurice, Rohin Kar, Benjamin Field, Hank Wu, Julian Wagner-Carena, David Aley, Joanna Lau, Zudi Lin, Donglai Wei, Hanspeter Pfister, Adi Peleg, Viren Jain, Jeff W. Lichtman

bioRxiv (2021)

A connectome and analysis of the adult Drosophila central brain

The neural circuits responsible for animal behavior remain largely unknown. We summarize new methods and present the circuitry of a large fraction of the brain of the fruit fly Drosophila melanogaster. Improved methods include new procedures to prepare, image, align, segment, find synapses in, and proofread such large data sets. We define cell types, refine computational compartments, and...

View details

A connectome and analysis of the adult Drosophila central brain

Louis K Scheffer, C Shan Xu, Michal Januszewski, Zhiyuan Lu, Shin-ya Takemura, Kenneth J Hayworth, Gary B Huang, Kazunori Shinomiya, Jeremy Maitlin-Shepard, Stuart Berg, Jody Clements, Philip M Hubbard, William T Katz, Lowell Umayam, Ting Zhao, David Ackerman, Tim Blakely, John Bogovic, Tom Dolafi, Dagmar Kainmueller, Takashi Kawase, Khaled A Khairy, Laramie Leavitt, Peter H Li, Larry Lindsey, Nicole Neubarth, Donald J Olbris, Hideo Otsuna, Eric T Trautman, Masayoshi Ito, Alexander S Bates, Jens Goldammer, Tanya Wolff, Robert Svirskas, Philipp Schlegel, Erika Neace, Christopher J Knecht, Chelsea X Alvarado, Dennis A Bailey, Samantha Ballinger, Jolanta A Borycz, Brandon S Canino, Natasha Cheatham, Michael Cook, Marisa Dreher, Octave Duclos, Bryon Eubanks, Kelli Fairbanks, Samantha Finley, Nora Forknall, Audrey Francis, Gary Patrick Hopkins, Emily M Joyce, SungJin Kim, Nicole A Kirk, Julie Kovalyak, Shirley Lauchie, Alanna Lohff, Charli Maldonado, Emily A Manley, Sari McLin, Caroline Mooney, Miatta Ndama, Omotara Ogundeyi, Nneoma Okeoma, Christopher Ordish, Nicholas Padilla, Christopher M Patrick, Tyler Paterson, Elliott E Phillips, Emily M Phillips, Neha Rampally, Caitlin Ribeiro, Madelaine K Robertson, Jon Thomson Rymer, Sean M Ryan, Megan Sammons, Anne K Scott, Ashley L Scott, Aya Shinomiya, Claire Smith, Kelsey Smith, Natalie L Smith, Margaret A Sobeski, Alia Suleiman, Jackie Swift, Satoko Takemura, Iris Talebi, Dorota Tarnogorska, Emily Tenshaw, Temour Tokhi, John J Walsh, Tansy Yang, Jane Anne Horne, Feng Li, Ruchi Parekh, Patricia K Rivlin, Vivek Jayaraman, Marta Costa, Gregory SXE Jefferis, Kei Ito, Stephan Saalfeld, Reed George, Ian Meinertzhagen, Gerald M Rubin, Harald F Hess, Viren Jain, Stephen M Plaza

eLife, vol. 9 (2020)

Denoising-based Image Compression for Connectomics

Connectomic reconstruction of neural circuits relies on nanometer resolution microscopy which produces on the order of a petabyte of imagery for each cubic millimeter of brain tissue. The cost of storing such data is a significant barrier to broadening the use of connectomic approaches and scaling to even larger volumes. We present an image compression approach that uses machine learning-based...

View details

Denoising-based Image Compression for Connectomics

David Charles Minnen, Michal Januszewski, Alex Shapson-Coe, Richard L. Schalek, Johannes Ballé, Jeff W. Lichtman, Viren Jain

bioRxiv (2021)

The Mind of a Mouse

Large scientific projects in genomics and astronomy are influential not because they answer any single ques- tion but because they enable investigation of continuously arising new questions from the same data-rich sources. Advances in automated mapping of the brain’s synaptic connections (connectomics) suggest that the complicated circuits underlying brain function are ripe for analysis. We...

View details

The Mind of a Mouse

Larry F. Abbott, Davi D. Bock, Edward M. Callaway, Winfried Denk, Catherine Dulac,, Adrienne L. Fairhall, Ila Fiete, Kristen M. Harris, Moritz Helmstaedter, Viren Jain, Narayanan Kasthuri, Yann LeCun, Jeff W. Lichtman, Peter B. Littlewood, Liqun Luo, John H.R. Maunsell, R. Clay Reid, Bruce R. Rosen, Gerald M. Rubin, Terrence J. Sejnowski, H. Sebastian Seung, Karel Svoboda, David W. Tank, Doris Tsao, David C. Van Essen

Cell, vol. 182 (2020)

An anatomical substrate of credit assignment in reinforcement learning

How is experience used to improve performance? In both biological and artificial systems, the optimization of parameters that affect behavior requires a process that determines whether a parameter affects the outcome and then modifies the parameter accordingly. Central to the recent bloom of artificial intelligence has been the error-backpropagation algorithm(Rumelhart, Hinton, and Williams...

View details

An anatomical substrate of credit assignment in reinforcement learning

Jorgen Kornfeld, Michal Januszewski, Michale S. Fee, Philipp Schubert, Viren Jain, Winfried Denk

bioRxiv (2020)

Automated Reconstruction of a Serial-Section EM Drosophila Brain with Flood-Filling Networks and Local Realignment

Reconstruction of neural circuitry at single-synapse resolution is an attractive target for improving understanding of the nervous system in health and disease. Serial section transmission electron microscopy (ssTEM) is among the most prolific imaging methods employed in pursuit of such reconstructions. We demonstrate how Flood-Filling Networks (FFNs) can be used to computationally segment a...

View details

Automated Reconstruction of a Serial-Section EM Drosophila Brain with Flood-Filling Networks and Local Realignment

Peter Li, Larry Lindsey, Michal Januszewski, Zhihao Zheng, Alexander Shakeel Bates, István Taisz, Mike Tyka, Matthew Nichols, Feng Li, Eric Perlman, Jeremy Maitin-Shepard, Tim Blakely, Laramie J Leavitt, Gregory S.X.E. Jefferis, Davi Bock, Viren Jain

bioRxiv (2019)

High-Precision Automated Reconstruction of Neurons with Flood-Filling Networks

Reconstruction of neural circuits from volume electron microscopy data requires the tracing of cells in their entirety, including all their neurites. Automated approaches have been developed for tracing, but their error rates are too high to generate reliable circuit diagrams without extensive human proofreading. We present flood-filling networks, a method for automated segmentation that,...

View details

High-Precision Automated Reconstruction of Neurons with Flood-Filling Networks

Michał Januszewski, Jörgen Kornfeld, Peter H. Li, Art Pope, Tim Blakely, Larry Lindsey, Jeremy B Maitin-Shepard, Mike Tyka, Winfried Denk, Viren Jain

Nature Methods (2018)

Superhuman Accuracy on the SNEMI3D Connectomics Challenge

For the past decade, convolutional networks have been used for 3D reconstruction of neurons from electron microscopic (EM) brain images. Recent years have seen great improvements in accuracy, as evidenced by submissions to the SNEMI3D benchmark challenge. Here we report the first submission to surpass the estimate of human accuracy provided by the SNEMI3D leaderboard. A variant of 3D UNet is...

View details

Superhuman Accuracy on the SNEMI3D Connectomics Challenge

Kisuk Lee, Jonathan Zung, Peter H. Li, Viren Jain, H. Sebastian Seung

arXiv, vol. abs/1706.00120 (2017)

Adversarial Image Alignment and Interpolation

Volumetric (3d) images are acquired for many scientific and biomedical purposes using imaging methods such as serial section microscopy, CT scans, and MRI. A frequent step in the analysis and reconstruction of such data is the alignment and registration of images that were acquired in succession along a spatial or temporal dimension. For example, in serial section electron microscopy,...

View details

Adversarial Image Alignment and Interpolation

Viren Jain

arXiv, vol. abs/1707.00067 (2017)

Combinatorial Energy Learning for Image Segmentation

We introduce a new machine learning approach for image segmentation that uses a neural network to model the conditional energy of a segmentation given an image. Our approach, combinatorial energy learning for image segmentation (CELIS) places a particular emphasis on modeling the inherent combinatorial nature of dense image segmentation problems. We propose efficient algorithms for learning...

View details

Combinatorial Energy Learning for Image Segmentation

Jeremy B. Maitin-Shepard, Viren Jain, Michal Januszewski, Peter H. Li, Pieter Abbeel

Advances in Neural Information Processing Systems 29 (2016)