Jump to Content
Viren Jain

Viren Jain

Authored Publications
Google Publications
Other Publications
Sort By
  • Title
  • Title, desc
  • Year
  • Year, desc
    Preview abstract Early machine-learning systems were inspired by neural networks — now AI might allow neuroscientists to get to grips with the brain’s unique complexities. View details
    Multi-Layered Maps of Neuropil with Segmentation Guided Contrastive Learning
    Sven Dorkenwald
    Daniel R. Berger
    Agnes L. Bodor
    Forrest Collman
    Casey M. Schneider-Mizell
    Nuno Maçarico da Costa
    Jeff W. Lichtman
    Nature Methods (2023)
    Preview abstract Maps of the nervous system that identify individual cells along with their type, subcellular components and connectivity have the potential to elucidate fundamental organizational principles of neural circuits. Nanometer-resolution imaging of brain tissue provides the necessary raw data, but inferring cellular and subcellular annotation layers is challenging. We present segmentation-guided contrastive learning of representations (SegCLR), a self-supervised machine learning technique that produces representations of cells directly from 3D imagery and segmentations. When applied to volumes of human and mouse cortex, SegCLR enables accurate classification of cellular subcompartments and achieves performance equivalent to a supervised approach while requiring 400-fold fewer labeled examples. SegCLR also enables inference of cell types from fragments as small as 10 μm, which enhances the utility of volumes in which many neurites are truncated at boundaries. Finally, SegCLR enables exploration of layer 5 pyramidal cell subtypes and automated large-scale analysis of synaptic partners in mouse visual cortex. View details
    Structured sampling of olfactory input by the fly mushroom body
    Zhihao Zheng
    Feng Li
    Corey Fisher
    Iqbal J. Ali
    Nadiya Sharifi
    Steven Calle-Schuler
    Joseph Hsu
    Najla Masoodpanah
    Lucia Kmecova
    Tom Kazimiers
    Eric Perlman
    Matthew Nichols
    Davi Bock
    Current Biology, vol. 32 (2022), pp. 3334-3349
    Preview abstract Associative memory formation and recall in the fruit fly Drosophila melanogaster is subserved by the mushroom body (MB). Upon arrival in the MB, sensory information undergoes a profound transformation from broadly tuned and stereotyped odorant responses in the olfactory projection neuron (PN) layer to narrowly tuned and nonstereotyped responses in the Kenyon cells (KCs). Theory and experiment suggest that this transformation is implemented by random connectivity between KCs and PNs. However, this hypothesis has been challenging to test, given the difficulty of mapping synaptic connections between large numbers of brain-spanning neurons. Here, we used a recent whole-brain electron microscopy volume of the adult fruit fly to map PN-to-KC connectivity at synaptic resolution. The PN-KC connectome revealed unexpected structure, with preponderantly food-responsive PN types converging at above-chance levels on downstream KCs. Axons of the overconvergent PN types tended to arborize near one another in the MB main calyx, making local KC dendrites more likely to receive input from those types. Overconvergent PN types preferentially co-arborize and connect with dendrites of αβ and α′β′ KC subtypes. Computational simulation of the observed network showed degraded discrimination performance compared with a random network, except when all signal flowed through the overconvergent, primarily food-responsive PN types. Additional theory and experiment will be needed to fully characterize the impact of the observed non-random network structure on associative memory formation and recall. View details
    SyConn2: dense synaptic connectivity inference for volume electron microscopy
    Philipp J. Schubert
    Sven Dorkenwald
    Jonathan Klimesch
    Fabian Svara
    Andrei Mancu
    Hashir Ahmad
    Michale S. Fee
    Joergen Kornfeld
    Nature Methods, vol. 19 (2022), 1367–1370
    Preview abstract The ability to acquire ever larger datasets of brain tissue using volume electron microscopy leads to an increasing demand for the automated extraction of connectomic information. We introduce SyConn2, an open-source connectome analysis toolkit, which works with both on-site high-performance compute environments and rentable cloud computing clusters. SyConn2 was tested on connectomic datasets with more than 10 million synapses, provides a web-based visualization interface and makes these data amenable to complex anatomical and neuronal connectivity queries. View details
    Preview abstract Connectomic reconstruction of neural circuits relies on nanometer resolution microscopy which produces on the order of a petabyte of imagery for each cubic millimeter of brain tissue. The cost of storing such data is a significant barrier to broadening the use of connectomic approaches and scaling to even larger volumes. We present an image compression approach that uses machine learning-based denoising and standard image codecs to compress raw electron microscopy imagery of neuropil up to 17-fold with negligible loss of reconstruction accuracy. View details
    A connectomic study of a petascale fragment of human cerebral cortex
    Alex Shapson-Coe
    Daniel R. Berger
    Yuelong Wu
    Richard L. Schalek
    Shuohong Wang
    Neha Karlupia
    Sven Dorkenwald
    Evelina Sjostedt
    Dongil Lee
    Luke Bailey
    Angerica Fitzmaurice
    Rohin Kar
    Benjamin Field
    Hank Wu
    Julian Wagner-Carena
    David Aley
    Joanna Lau
    Zudi Lin
    Donglai Wei
    Hanspeter Pfister
    Adi Peleg
    Jeff W. Lichtman
    bioRxiv (2021)
    Preview abstract We acquired a rapidly preserved human surgical sample from the temporal lobe of the cerebral cortex. We stained a 1 mm3 volume with heavy metals, embedded it in resin, cut more than 5000 slices at ∼30 nm and imaged these sections using a high-speed multibeam scanning electron microscope. We used computational methods to render the three-dimensional structure containing 57,216 cells, hundreds of millions of neurites and 133.7 million synaptic connections. The 1.4 petabyte electron microscopy volume, the segmented cells, cell parts, blood vessels, myelin, inhibitory and excitatory synapses, and 104 manually proofread cells are available to peruse online. Many interesting and unusual features were evident in this dataset. Glia outnumbered neurons 2:1 and oligodendrocytes were the most common cell type in the volume. Excitatory spiny neurons comprised 69% of the neuronal population, and excitatory synapses also were in the majority (76%). The synaptic drive onto spiny neurons was biased more strongly toward excitation (70%) than was the case for inhibitory interneurons (48%). Despite incompleteness of the automated segmentation caused by split and merge errors, we could automatically generate (and then validate) connections between most of the excitatory and inhibitory neuron types both within and between layers. In studying these neurons we found that deep layer excitatory cell types can be classified into new subsets, based on structural and connectivity differences, and that chandelier interneurons not only innervate excitatory neuron initial segments as previously described, but also each other’s initial segments. Furthermore, among the thousands of weak connections established on each neuron, there exist rarer highly powerful axonal inputs that establish multi-synaptic contacts (up to ∼20 synapses) with target neurons. Our analysis indicates that these strong inputs are specific, and allow small numbers of axons to have an outsized role in the activity of some of their postsynaptic partners. View details
    An anatomical substrate of credit assignment in reinforcement learning
    Jorgen Kornfeld
    Michale S. Fee
    Philipp Schubert
    Winfried Denk
    bioRxiv (2020)
    Preview abstract How is experience used to improve performance? In both biological and artificial systems, the optimization of parameters that affect behavior requires a process that determines whether a parameter affects the outcome and then modifies the parameter accordingly. Central to the recent bloom of artificial intelligence has been the error-backpropagation algorithm(Rumelhart, Hinton, and Williams 1986) , which computationally retraces the signal from the output to each synapse (weight) and allows a large number of parameters to be optimized in parallel at high learning rates. Biological systems, however, lack an obvious mechanism to retrace the signal path. Here we show, by combining high-throughput volume electron microscopy (Denk and Horstmann 2004) and automated connectomic analysis(Januszewski et al. 2018; Dorkenwald et al. 2017; Schubert et al. 2019) , that the synaptic architecture of songbird basal ganglia supports a form of local credit assessment proposed in a model of songbird reinforcement learning (M. S. Fee and Goldberg 2011). We show that three of this model’s major predictions hold true: first, cortical axons that encode exploratory motor variability terminate predominantly on dendritic shafts of spiny neurons. Second, cortical axons that encode timing seek out spines, which enable calcium-based coincidence detection (R. Yuste and Denk 1995) and appear to be capable of creating and storing eligibility traces (Yagishita et al. 2014). Third, synapse pairs that presynaptically share a cortical timing axon and post-synaptically a medium spiny dendrite are substantially more similar in size than expected, indicating a history of Hebbian plasticity (Bartol et al. 2015; Kasthuri et al. 2015) . Combined with numerical simulations these data provide strong evidence for a model of basal ganglia learning with a biologically plausible credit assignment mechanism. View details
    The Mind of a Mouse
    Larry F. Abbott
    Davi D. Bock
    Edward M. Callaway
    Winfried Denk
    Catherine Dulac,
    Adrienne L. Fairhall
    Ila Fiete
    Kristen M. Harris
    Moritz Helmstaedter
    Narayanan Kasthuri
    Yann LeCun
    Jeff W. Lichtman
    Peter B. Littlewood
    Liqun Luo
    John H.R. Maunsell
    R. Clay Reid
    Bruce R. Rosen
    Gerald M. Rubin
    Terrence J. Sejnowski
    H. Sebastian Seung
    Karel Svoboda
    David W. Tank
    Doris Tsao
    David C. Van Essen
    Cell, vol. 182 (2020)
    Preview abstract Large scientific projects in genomics and astronomy are influential not because they answer any single ques- tion but because they enable investigation of continuously arising new questions from the same data-rich sources. Advances in automated mapping of the brain’s synaptic connections (connectomics) suggest that the complicated circuits underlying brain function are ripe for analysis. We discuss benefits of mapping a mouse brain at the level of synapses. View details
    A connectome and analysis of the adult Drosophila central brain
    Louis K Scheffer
    C Shan Xu
    Zhiyuan Lu
    Shin-ya Takemura
    Kenneth J Hayworth
    Gary B Huang
    Kazunori Shinomiya
    Stuart Berg
    Jody Clements
    Philip M Hubbard
    William T Katz
    Lowell Umayam
    Ting Zhao
    David Ackerman
    John Bogovic
    Tom Dolafi
    Dagmar Kainmueller
    Takashi Kawase
    Khaled A Khairy
    Larry Lindsey
    Nicole Neubarth
    Donald J Olbris
    Hideo Otsuna
    Eric T Trautman
    Masayoshi Ito
    Alexander S Bates
    Jens Goldammer
    Tanya Wolff
    Robert Svirskas
    Philipp Schlegel
    Erika Neace
    Christopher J Knecht
    Chelsea X Alvarado
    Dennis A Bailey
    Samantha Ballinger
    Jolanta A Borycz
    Brandon S Canino
    Natasha Cheatham
    Michael Cook
    Marisa Dreher
    Octave Duclos
    Bryon Eubanks
    Kelli Fairbanks
    Samantha Finley
    Nora Forknall
    Audrey Francis
    Gary Patrick Hopkins
    Emily M Joyce
    SungJin Kim
    Nicole A Kirk
    Julie Kovalyak
    Shirley Lauchie
    Alanna Lohff
    Charli Maldonado
    Emily A Manley
    Sari McLin
    Caroline Mooney
    Miatta Ndama
    Omotara Ogundeyi
    Nneoma Okeoma
    Christopher Ordish
    Nicholas Padilla
    Christopher M Patrick
    Tyler Paterson
    Elliott E Phillips
    Emily M Phillips
    Neha Rampally
    Caitlin Ribeiro
    Madelaine K Robertson
    Jon Thomson Rymer
    Sean M Ryan
    Megan Sammons
    Anne K Scott
    Ashley L Scott
    Aya Shinomiya
    Claire Smith
    Kelsey Smith
    Natalie L Smith
    Margaret A Sobeski
    Alia Suleiman
    Jackie Swift
    Satoko Takemura
    Iris Talebi
    Dorota Tarnogorska
    Emily Tenshaw
    Temour Tokhi
    John J Walsh
    Tansy Yang
    Jane Anne Horne
    Feng Li
    Ruchi Parekh
    Patricia K Rivlin
    Vivek Jayaraman
    Marta Costa
    Gregory SXE Jefferis
    Kei Ito
    Stephan Saalfeld
    Reed George
    Ian Meinertzhagen
    Gerald M Rubin
    Harald F Hess
    Stephen M Plaza
    eLife, vol. 9 (2020)
    Preview abstract The neural circuits responsible for animal behavior remain largely unknown. We summarize new methods and present the circuitry of a large fraction of the brain of the fruit fly Drosophila melanogaster. Improved methods include new procedures to prepare, image, align, segment, find synapses in, and proofread such large data sets. We define cell types, refine computational compartments, and provide an exhaustive atlas of cell examples and types, many of them novel. We provide detailed circuits consisting of neurons and their chemical synapses for most of the central brain. We make the data public and simplify access, reducing the effort needed to answer circuit questions, and provide procedures linking the neurons defined by our analysis with genetic reagents. Biologically, we examine distributions of connection strengths, neural motifs on different scales, electrical consequences of compartmentalization, and evidence that maximizing packing density is an important criterion in the evolution of the fly's brain. View details
    Preview abstract Recent advances in 3d electron microscopy are yielding ever larger reconstructions of brain tissue, encompassing thousands of individual neurons interconnected by millions of synapses. Interpreting reconstructions at this scale demands advances in the automated analysis of neuronal morphologies, for example by identifying morphological and functional subcompartments within neurons. We present a method that for the first time uses full 3d input (voxels) to automatically classify reconstructed neuron fragments as axon, dendrite, or somal subcompartments. Based on 3d convolutional neural networks, this method achieves a mean f1-score of 0.972, exceeding the previous state of the art of 0.955. The resulting predictions can support multiple analysis and proofreading applications. In particular, we leverage finely localized subcompartment predictions for automated detection and correction of merge errors in the volume reconstruction, successfully detecting 90.6% of inter-class merge errors with a false positive rate of only 2.7%. View details
    Preview abstract Reconstruction of neural circuitry at single-synapse resolution is an attractive target for improving understanding of the nervous system in health and disease. Serial section transmission electron microscopy (ssTEM) is among the most prolific imaging methods employed in pursuit of such reconstructions. We demonstrate how Flood-Filling Networks (FFNs) can be used to computationally segment a forty-teravoxel whole-brain Drosophila ssTEM volume. To compensate for data irregularities and imperfect global alignment, FFNs were combined with procedures that locally re-align serial sections as well as dynamically adjust and synthesize image content. The proposed approach produced a largely merger-free segmentation of the entire ssTEM Drosophila brain, which we make freely available. As compared to manual tracing using an efficient skeletonization strategy, the segmentation enabled circuit reconstruction and analysis workflows that were an order of magnitude faster. View details
    Preview abstract Reconstruction of neural circuits from volume electron microscopy data requires the tracing of cells in their entirety, including all their neurites. Automated approaches have been developed for tracing, but their error rates are too high to generate reliable circuit diagrams without extensive human proofreading. We present flood-filling networks, a method for automated segmentation that, similar to most previous efforts, uses convolutional neural networks, but contains in addition a recurrent pathway that allows the iterative optimization and extension of individual neuronal processes. We used flood-filling networks to trace neurons in a dataset obtained by serial block-face electron microscopy of a zebra finch brain. Using our method, we achieved a mean error-free neurite path length of 1.1 mm, and we observed only four mergers in a test set with a path length of 97 mm. The performance of flood-filling networks was an order of magnitude better than that of previous approaches applied to this dataset, although with substantially increased computational costs. View details
    Preview abstract Algorithmic reconstruction of neurons from volume electron microscopy data traditionally requires training machine learning models on dataset-specific ground truth annotations that are expensive and tedious to acquire. We enhanced the training procedure of an unsupervised image-to-image translation method with additional components derived from an automated neuron segmentation approach. We show that this method, Segmentation-Enhanced CycleGAN, enables near perfect reconstruction accuracy on a benchmark connectomics segmentation dataset despite operating in a “zero-shot” setting in which only volumetric labels from a different volume imaging method were used. View details
    Preview abstract Volumetric (3d) images are acquired for many scientific and biomedical purposes using imaging methods such as serial section microscopy, CT scans, and MRI. A frequent step in the analysis and reconstruction of such data is the alignment and registration of images that were acquired in succession along a spatial or temporal dimension. For example, in serial section electron microscopy, individual 2d sections are imaged via electron microscopy and then must be aligned to one another in order to produce a coherent 3d volume. State of the art approaches find image correspondences derived from patch matching and invariant feature detectors, and then solve optimization problems that rigidly or elastically deform series of images into an aligned volume. Here we show how fully convolutional neural networks trained with an adversarial loss function can be used for two tasks: (1) synthesis of missing or damaged image data from adjacent sections, and (2) fine-scale alignment of block-face electron microscopy data. Finally, we show how these two capabilities can be combined in order to produce artificial isotropic volumes from anisotropic image volumes using a super-resolution adversarial alignment and interpolation approach. View details
    Superhuman Accuracy on the SNEMI3D Connectomics Challenge
    Kisuk Lee
    Jonathan Zung
    H. Sebastian Seung
    arXiv, vol. abs/1706.00120 (2017)
    Preview abstract For the past decade, convolutional networks have been used for 3D reconstruction of neurons from electron microscopic (EM) brain images. Recent years have seen great improvements in accuracy, as evidenced by submissions to the SNEMI3D benchmark challenge. Here we report the first submission to surpass the estimate of human accuracy provided by the SNEMI3D leaderboard. A variant of 3D UNet is trained on a primary task of predicting affinities between nearest neighbor voxels, and an auxiliary task of predicting long-range affinities. The training data is augmented by simulated image defects. The nearest neighbor affinities are used to create an oversegmentation, and then supervoxels are greedily agglomerated based on mean affinity. The resulting SNEMI3D score exceeds the estimate of human accuracy by a large margin. While one should be cautious about extrapolating from the SNEMI3D benchmark to real-world accuracy of large-scale neural circuit reconstruction, our submission inspires optimism that the goal of full automation may be realizable in the future. View details
    Solving large Multicut problems for connectomics via domain decomposition
    Constantin Pape
    Thorsten Beier
    Davi Bock
    Anna Kreshuk
    ICCV Bioimage Computing Workshop (2017)
    Preview abstract In this contribution we demonstrate how a Multicut-based segmentation pipeline can be scaled up to datasets of hundreds of Gigabytes in size. Such datasets are prevalent in connectomics, where neuron segmentation needs to be performed across very large electron microscopy image volumes. We show the advantages of a hierarchical block-wise scheme over local stitching strategies and evaluate the performance of different Multicut solvers for the segmentation of the blocks in the hierarchy. We validate the accuracy of our algorithm on a small fully annotated dataset (5×5×5 μm) and demonstrate no significant loss in segmentation quality compared to solving the Multicut problem globally. We evaluate the scalability of the algorithm on a 95×60×60 μm image volume and show that solving the Multicut problem is no longer the bottleneck of the segmentation pipeline. View details
    Preview abstract State-of-the-art image segmentation algorithms generally consist of at least two successive and distinct computations: a boundary detection process that uses local image information to classify image locations as boundaries between objects, followed by a pixel grouping step such as watershed or connected components that clusters pixels into segments. Prior work has varied the complexity and approach employed in these two steps, including the incorporation of multi-layer neural networks to perform boundary prediction, and the use of global optimizations during pixel clustering. We propose a unified and end-to-end trainable machine learning approach, flood-filling networks, in which a recurrent 3d convolutional network directly produces individual segments from a raw image. The proposed approach robustly segments images with an unknown and variable number of objects as well as highly variable object sizes. We demonstrate the approach on a challenging 3d image segmentation task, connectomic reconstruction from volume electron microscopy data, on which flood-filling neural networks substantially improve accuracy over other state-of-the-art methods. The proposed approach can replace complex multi-step segmentation pipelines with a single neural network that is learned end-to-end. View details
    Preview abstract We introduce a new machine learning approach for image segmentation that uses a neural network to model the conditional energy of a segmentation given an image. Our approach, combinatorial energy learning for image segmentation (CELIS) places a particular emphasis on modeling the inherent combinatorial nature of dense image segmentation problems. We propose efficient algorithms for learning deep neural networks to model the energy function, and for local optimization of this energy in the space of supervoxel agglomerations. We extensively evaluate our method on a publicly available 3-D microscopy dataset with 25 billion voxels of ground truth data. On an 11 billion voxel test set, we find that our method improves volumetric reconstruction accuracy by more than 20% as compared to two state-of-the-art baseline methods: graph-based segmentation of the output of a 3-D convolutional neural network trained to predict boundaries, as well as a random forest classifier trained to agglomerate supervoxels that were generated by a 3-D convolutional neural network. View details
    No Results Found