Jump to Content
Klaus-Robert Müller

Klaus-Robert Müller

Klaus-Robert Müller has been a professor of computer science at Technische Universität Berlin since 2006; at the same time he is directing rsp. co-directing the Berlin Machine Learning Center and the Berlin Big Data Center and most recently BIFOLD . He studied physics in Karlsruhe from1984 to 1989 and obtained his Ph.D. degree in computer science at Technische Universität Karlsruhe in 1992. After completing a postdoctoral position at GMD FIRST in Berlin, he was a research fellow at the University of Tokyo from 1994 to 1995. In 1995, he founded the Intelligent Data Analysis group at GMD-FIRST (later Fraunhofer FIRST) and directed it until 2008. From 1999 to 2006, he was a professor at the University of Potsdam. From 2012 he has been Distinguished Professor at Korea University in Seoul. In 2020/2021 he spent his sabbatical at Google Brain as a Principal Scientist. Among others, he was awarded the Olympus Prize for Pattern Recognition (1999), the SEL Alcatel Communication Award (2006), the Science Prize of Berlin by the Governing Mayor of Berlin (2014), the Vodafone Innovations Award (2017), Hector Science Award (2024), Pattern Recognition Best Paper award (2020), Digital Signal Processing Best Paper award (2022). In 2012, he was elected member of the German National Academy of Sciences-Leopoldina, in 2017 of the Berlin Brandenburg Academy of Sciences, in 2021 of the German National Academy of Science and Engineering and also in 2017 external scientific member of the Max Planck Society. From 2019 on he became an ISI Highly Cited researcher in the cross-disciplinary area. His research interests are intelligent data analysis and Machine Learning in the sciences (Neuroscience (specifically Brain-Computer Interfaces, Physics, Chemistry) and in industry.
Authored Publications
Google Publications
Other Publications
Sort By
  • Title
  • Title, desc
  • Year
  • Year, desc
    Accurate global machine learning force fields for molecules with hundreds of atoms
    Stefan Chmiela
    Valentin Vassilev Galindo
    Adil Kabylda
    Huziel E. Sauceda
    Alexandre Tkatchenko
    Science Advances, vol. 9(2) (2023), eadf0873
    Preview abstract Global machine learning force fields, with the capacity to capture collective interactions in molecular systems, now scale up to a few dozen atoms due to considerable growth of model complexity with system size. For larger molecules, locality assumptions are introduced, with the consequence that nonlocal interactions are not described. Here, we develop an exact iterative approach to train global symmetric gradient domain machine learning (sGDML) force fields (FFs) for several hundred atoms, without resorting to any potentially uncontrolled approximations. All atomic degrees of freedom remain correlated in the global sGDML FF, allowing the accurate description of complex molecules and materials that present phenomena with far-reaching characteristic correlation lengths. We assess the accuracy and efficiency of sGDML on a newly developed MD22 benchmark dataset containing molecules from 42 to 370 atoms. The robustness of our approach is demonstrated in nanosecond path-integral molecular dynamics simulations for supramolecular complexes in the MD22 dataset. View details
    Canonical Response Parameterization: Quantifying the structure of responses to single-pulse intracranial electrical brain stimulation
    Kai J. Miller
    Gabriela Ojeda Valencia
    Harvey Huang
    Nicholas M. Gregg
    Gregory A. Worrell
    Dora Hermes
    Plos Computational Biology, vol. 19(5) (2023), e1011105
    Preview abstract Single-pulse electrical stimulation in the nervous system, often called cortico-cortical evoked potential (CCEP) measurement, is an important technique to understand how brain regions interact with one another. Voltages are measured from implanted electrodes in one brain area while stimulating another with brief current impulses separated by several seconds. Historically, researchers have tried to understand the significance of evoked voltage polyphasic deflections by visual inspection, but no general-purpose tool has emerged to understand their shapes or describe them mathematically. We describe and illustrate a new technique to parameterize brain stimulation data, where voltage response traces are projected into one another using a semi-normalized dot product. The length of timepoints from stimulation included in the dot product is varied to obtain a temporal profile of structural significance, and the peak of the profile uniquely identifies the duration of the response. Using linear kernel PCA, a canonical response shape is obtained over this duration, and then single-trial traces are parameterized as a projection of this canonical shape with a residual term. Such parameterization allows for dissimilar trace shapes from different brain areas to be directly compared by quantifying cross-projection magnitudes, response duration, canonical shape projection amplitudes, signal-to-noise ratios, explained variance, and statistical significance. Artifactual trials are automatically identified by outliers in sub-distributions of cross-projection magnitude, and rejected. This technique, which we call “Canonical Response Parameterization” (CRP) dramatically simplifies the study of CCEP shapes, and may also be applied in a wide range of other settings involving event-triggered data. View details
    Higher-Order Explanations of Graph Neural Networks via Relevant Walks
    Thomas Schnake
    Oliver Eberle
    Jonas Lederer
    Shin Nakajima
    Kristof T. Schütt
    Gregoire Montavon
    IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44(11) (2022), pp. 7581 - 7596
    Preview abstract Graph Neural Networks (GNNs) are a popular approach for predicting graph structured data. As GNNs tightly entangle the input graph into the neural network structure, common explainable AI approaches are not applicable. To a large extent, GNNs have remained black-boxes for the user so far. In this paper, we show that GNNs can in fact be naturally explained using higher-order expansions, i.e., by identifying groups of edges that jointly contribute to the prediction. Practically, we find that such explanations can be extracted using a nested attribution scheme, where existing techniques such as layer-wise relevance propagation (LRP) can be applied at each step. The output is a collection of walks into the input graph that are relevant for the prediction. Our novel explanation method, which we denote by GNN-LRP, is applicable to a broad range of graph neural networks and lets us extract practically relevant insights on sentiment analysis of text data, structure-property relationships in quantum chemistry, and image classification. View details
    Preview abstract The application of machine learning (ML) methods in quantum chemistry has enabled the study of numerous chemical phenomena, which are computationally intractable with traditional ab initio methods. However, some quantum mechanical properties of molecules and materials depend on non-local electronic effects, which are often neglected due to the difficulty of modelling them efficiently. This work proposes a modified attention mechanism adapted to the underlying physics, which allows to recover the relevant non-local effects. Namely, we introduce spherical harmonic coordinates (SPHCs) to reflect higher order geometric information for each atom in a molecule, enabling a non-local formulation of attention in the SPHC space. Our proposed model So3krates -- a self-attention based message passing neural network (MPNN) -- uncouples geometric information from atomic features, making them independently amenable to attention mechanisms. We show that in contrast to other published methods, So3krates is able to describe quantum mechanical effects due to orbital overlap over arbitrary length scales. Further, So3krates is shown to match or exceed state-of-the-art performance on the popular MD-17 and QM-7X benchmarks, notably, requiring a significantly lower number of parameters while at the same time giving a substantial speedup compared to other models. View details
    Toward Explainable Artificial Intelligence for Regression Models: A methodological perspective
    Simon Letzgus
    Jonas Lederer
    Wojciech Samek
    Gregoire Montavon
    IEEE Signal Processing Magazine, vol. 39 (4) (2022), 40–58
    Preview abstract In addition to the impressive predictive power of machine learning (ML) models, more recently, explanation methods have emerged that enable an interpretation of complex nonlinear learning models, such as deep neural networks. Gaining a better understanding is especially important, e.g., for safety-critical ML applications or medical diagnostics and so on. Although such explainable artificial intelligence (XAI) techniques have reached significant popularity for classifiers, thus far, little attention has been devoted to XAI for regression models (XAIR). In this review, we clarify the fundamental conceptual differences of XAI for regression and classification tasks, establish novel theoretical insights and analysis for XAIR, provide demonstrations of XAIR on genuine practical regression problems, and finally, discuss challenges remaining for the field. View details
    BIGDML—Towards accurate quantum machine learning force fields for materials
    Huziel Sauceda
    Luis Gálvez-González
    Stefan Chmiela
    Lauro Oliver Paz Borbon
    Alexandre Tkatchenko
    Nature Communications, vol. 13 (2022), pp. 3733
    Preview abstract Machine-learning force fields (MLFF) should be accurate, computationally and data efficient, and applicable to molecules, materials, and interfaces thereof. Currently, MLFFs often introduce tradeoffs that restrict their practical applicability to small subsets of chemical space or require exhaustive datasets for training. Here, we introduce the Bravais-Inspired Gradient-Domain Machine Learning (BIGDML) approach and demonstrate its ability to construct reliable force fields using a training set with just 10–200 geometries for materials including pristine and defect-containing 2D and 3D semiconductors and metals, as well as chemisorbed and physisorbed atomic and molecular adsorbates on surfaces. The BIGDML model employs the full relevant symmetry group for a given material, does not assume artificial atom types or localization of atomic interactions and exhibits high data efficiency and state-of-the-art energy accuracies (errors substantially below 1 meV per atom) for an extended set of materials. Extensive path-integral molecular dynamics carried out with BIGDML models demonstrate the counterintuitive localization of benzene–graphene dynamics induced by nuclear quantum effects and their strong contributions to the hydrogen diffusion coefficient in a Pd crystal for a wide range of temperatures. View details
    Artificial Intelligence and Pathology: from Principles to Practice and Future Applications in Histomorphology and Molecular Profiling
    Albrecht Stenzinger
    Max Alber
    Michael Allgäuer
    Phillip Jurmeister
    Michael Bockmayr
    Jan Budczies
    Jochen Lennerz
    Johannes Eschrich
    Daniel Kazdal
    Peter Schirmacher
    Alex H Wagner
    Frank Tacke
    David Capper
    Frederick Klauschen
    Seminars in Cancer Biology, vol. 84 (2022), pp. 129-143
    Preview abstract The complexity of diagnostic (surgical) pathology has increased substantially over the last decades with respect to histomorphological and molecular profiling and has steadily expanded its role in tumor diagnostics and beyond from disease entity identification via prognosis estimation to precision therapy prediction. It is therefore not surprising that pathology is among the disciplines in medicine with high expectations in the application of artificial intelligence (AI) or machine learning approaches given its capabilities to analyse complex data in a quantitative and standardised manner to further enhance scope and precision of diagnostics. While an obvious application is the analysis of histological images, recent applications for the analysis of molecular profiling data from different sources and clinical data support the notion that AI will support both histopathology and molecular pathology in the future. At the same time, current literature should not be misunderstood in a way that pathologists will likely be replaced by AI applications in the foreseeable future. Although AI will likely transform pathology in the coming years, recent studies reporting AI algorithms to diagnose cancer or predict certain molecular properties deal with relatively simple diagnostic problems that fall short of the diagnostic complexity pathologists face in clinical routine. Here, we review the pertinent literature of AI methods and their applications to pathology, and put the current achievements and what can be expected in the future in the context of the requirements for research and routine diagnostics. View details
    Super-resolution in Molecular Dynamics Trajectory Reconstruction with Bi-Directional Neural Networks
    Paul Ludwig Winkler
    Huziel Saucceda
    Machine Learning: Science and Technology, vol. 3 (2022), pp. 025011
    Preview abstract Molecular dynamics (MD) simulations are a cornerstone in science, enabling the investigation of a system’s thermodynamics all the way to analyzing intricate molecular interactions. In general, creating extended molecular trajectories can be a computationally expensive process, for example, when running ab-initio simulations. Hence, repeating such calculations to either obtain more accurate thermodynamics or to get a higher resolution in the dynamics generated by a fine-grained quantum interaction can be time- and computational resource-consuming. In this work, we explore different machine learning methodologies to increase the resolution of MD trajectories on-demand within a post-processing step. As a proof of concept, we analyse the performance of bi-directional neural networks (NNs) such as neural ODEs, Hamiltonian networks, recurrent NNs and long short-term memories, as well as the uni-directional variants as a reference, for MD simulations (here: the MD17 dataset). We have found that Bi-LSTMs are the best performing models; by utilizing the local time-symmetry of thermostated trajectories they can even learn long-range correlations and display high robustness to noisy dynamics across molecular complexity. Our models can reach accuracies of up to 10−4 Å in trajectory interpolation, which leads to the faithful reconstruction of several unseen high-frequency molecular vibration cycles. This renders the comparison between the learned and reference trajectories indistinguishable. The results reported in this work can serve (1) as a baseline for larger systems, as well as (2) for the construction of better MD integrators. View details
    Algorithmic Differentiation for Automatized Modelling of Machine Learned Force Fields
    Niklas Schmitz
    Stefan Chmiela
    The Journal of Physical Chemistry Letters, vol. 13(43) (2022), pp. 10183-10189
    Preview abstract Reconstructing force fields (FFs) from atomistic simulation data is a challenge since accurate data can be highly expensive. Here, machine learning (ML) models can help to be data economic as they can be successfully constrained using the underlying symmetry and conservation laws of physics. However, so far, every descriptor newly proposed for an ML model has required a cumbersome and mathematically tedious remodeling. We therefore propose using modern techniques from algorithmic differentiation within the ML modeling process, effectively enabling the usage of novel descriptors or models fully automatically at an order of magnitude higher computational efficiency. This paradigmatic approach enables not only a versatile usage of novel representations and the efficient computation of larger systems─all of high value to the FF community─but also the simple inclusion of further physical knowledge, such as higher-order information (e.g., Hessians, more complex partial differential equations constraints etc.), even beyond the presented FF domain. View details
    Harmoni: a Method for Eliminating Spurious Interactions due to the Harmonic Components in Neuronal Data
    Mina Jamshidi Idaji
    Juanli Zhang
    Tilman Stephani
    Guido Nolte
    Arno Villringer
    Vadim Nikulin
    Neuroimage, vol. 252 (2022), pp. 119053
    Preview abstract Cross-frequency synchronization (CFS) has been proposed as a mechanism for integrating spatially and spectrally distributed information in the brain. However, investigating CFS in Magneto- and Electroencephalography (MEG/EEG) is hampered by the presence of spurious neuronal interactions due to the non-sinusoidal waveshape of brain oscillations. Such waveshape gives rise to the presence of oscillatory harmonics mimicking genuine neuronal oscillations. Until recently, however, there has been no methodology for removing these harmonics from neuronal data. In order to address this long-standing challenge, we introduce a novel method (called HARMOnic miNImization - Harmoni) that removes the signal components which can be harmonics of a non-sinusoidal signal. Harmoni’s working principle is based on the presence of CFS between harmonic components and the fundamental component of a non-sinusoidal signal. We extensively tested Harmoni in realistic EEG simulations. The simulated couplings between the source signals represented genuine and spurious CFS and within-frequency phase synchronization. Using diverse evaluation criteria, including ROC analyses, we showed that the within- and cross-frequency spurious interactions are suppressed significantly, while the genuine activities are not affected. Additionally, we applied Harmoni to real resting-state EEG data revealing intricate remote connectivity patterns which are usually masked by the spurious connections. Given the ubiquity of non-sinusoidal neuronal oscillations in electrophysiological recordings, Harmoni is expected to facilitate novel insights into genuine neuronal interactions in various research fields, and can also serve as a steppingstone towards the development of further signal processing methods aiming at refining within- and cross-frequency synchronization in electrophysiological recordings. View details
    Efficient Computation of Higher-Order Subgraph Attribution via Message Passing
    Ping Xiong
    Thomas Schnake
    Gregoire Montavon
    Shin Nakajima
    ICML (2022) (to appear)
    Preview abstract Explaining graph neural networks (GNNs) has become more and more important recently. Higherorder interpretation schemes, such as GNNLRP (layer-wise relevance propagation for GNN), emerged as powerful tools for unraveling how different features interact thereby contributing to explaining GNNs. Methods such as GNN-LRP perform walks between nodes at each layer, and there are exponentially many such walks. In this work, we demonstrate that such exponential complexity can be avoided, in particular, we propose novel linear-time (w.r.t. depth) algorithms that enable to efficiently perform GNN-LRP for subgraphs. Our algorithms are derived via message passing techniques that make use of the distributive property, thereby directly computing quantities for higher-order explanations. We further adapt our efficient algorithms to compute a generalization of subgraph attributions that also takes into account the neighboring graph features. Experimental results show significant acceleration of the proposed algorithms and demonstrate a high usefulness and scalability of our novel generalized subgraph attribution. View details
    Towards Robust Explanations for Deep Neural Networks
    Ann-Kathrin Dombrowski
    Christopher Johannes Anders
    Pan Kessel
    Pattern Recognition, vol. 121 (2022), pp. 108194
    Preview abstract Explanation methods shed light on the decision process of black-box classifiers such as deep neural networks. But their usefulness can be compromised because they are susceptible to manipulations. With this work, we aim to enhance the resilience of explanations. We develop a unified theoretical framework for deriving bounds on the maximal manipulability of a model. Based on these theoretical insights, we present three different techniques to boost robustness against manipulation: training with weight decay, smoothing activation functions, and minimizing the Hessian of the network. Our experimental results confirm the effectiveness of these approaches. View details
    A Unifying Review of Deep and Shallow Anomaly Detection
    Lukas Ruff
    Jacob Reinhard Kauffmann
    Robert Vandermeulen
    Gregoire Montavon
    Wojciech Samek
    Marius Kloft
    Thomas G. Dietterich
    Proc of the IEEE, vol. 109(5) (2021), pp. 756-795 (to appear)
    Preview abstract Deep learning approaches to anomaly detection have recently improved the state of the art in detection performance on complex datasets such as large collections of images or text. These results have sparked a renewed interest in the anomaly detection problem and led to the introduction of a great variety of new methods. With the emergence of numerous such methods that include approaches based on generative models, one-class classification, and reconstruction, there is a growing need to bring methods of this field into a systematic and unified perspective. In this review, we therefore aim to identify the common underlying principles as well as the assumptions that are often made implicitly by various methods. In particular, we draw connections between classic ‘shallow’ and novel deep approaches and show how they exactly relate and moreover how this relation might cross-fertilize or extend both directions. We further provide an empirical assessment of major existing methods that is enriched by the use of recent explainability techniques, and present specific worked-through examples together with practical advice. Finally, we outline critical open challenges and identify specific paths for future research in anomaly detection. View details
    Combining Machine Learning and Computational Chemistry for Predictive Insights Into Chemical Systems
    John A Keith
    Valentin Vassilev Galindo
    Bingqing Cheng
    Stefan Chmiela
    Michael Gastegger
    Alexandre Tkatchenko
    Chemical Reviews, vol. 121 (16) (2021), 9816-9872, https://pubs.acs.org/doi/pdf/10.1021/acs.chemrev.1c00107
    Preview abstract Machine learning models are poised to make transformative impact in the chemical sciences by dramatically accelerating computational algorithms and amplifying insights available from computational chemistry methods. However, achieving this requires a confluence and coaction of expertise in computer science and physical sciences. This review is written for new and experienced researchers working at the intersection of both fields. We first provide concise tutorials of computational chemistry, machine learning methods, and how insights involving both can be achieved. We then follow with a critical review of noteworthy applications that demonstrate how computational quantum chemistry and machine learning can be used together to provide insightful (and useful) predictions in molecular and materials modeling, retrosyntheses, catalysis, and drug design. View details
    Dynamical Strengthening of Covalent and Non-Covalent Molecular Interactions by Nuclear Quantum Effects at Finite Temperature
    Huziel Saucceda
    Stefan Chmiela
    Valentin Vassilev Galindo
    Alexandre Tkatchenko
    Nature Communications, vol. 12 (2021), pp. 442
    Preview abstract Nuclear quantum effects (NQE) tend to generate delocalized molecular dynamics due to the anharmonicity of interatomic interactions. Here, we present evidence that NQE often enhance electronic interactions and, in turn, can result in dynamical molecular stabilization at finite temperature. The underlying physical mechanism promoted by NQE depends on the particular interaction under consideration. First, the effective reduction of interatomic distances between functional groups within a molecule enhances the n → π ∗ interaction by increasing the overlap between molecular orbitals or by strengthening electrostatic interactions between neighboring charge densities. Second, NQE can localize methyl rotors by temporarily changing molecular bond orders and leading to the emergence of localized transient rotor states. Third, for noncovalent interactions the strengthening comes from the increase of the polarizability given the expanded average interatomic distances induced by NQE. The implications of these boosted interactions include counterintuitive hydroxyl–hydroxyl bonding, hindered methyl rotor dynamics, and molecular stiffening which generates smoother free-energy surfaces. These results challenge the general assumption that NQE tend to mainly generate delocalized dynamics and reveal that NQE also play an active role in dynamical strengthening of molecular interactions. Our findings yield new insights into the versatile role of nuclear quantum fluctuations in molecules and materials View details
    Basis profile curve identification to understand electrical stimulation effects in human brain networks
    Kai Joshua Miller
    Dora Hermes
    Plos Computational Biology, vol. 17(9) (2021), e1008710, https://doi.org/10.1371/journal.pcbi.1008710
    Preview abstract Brain networks can be explored by delivering brief pulses of electrical current in one area while measuring voltage responses in other areas. We propose a convergent paradigm to study brain dynamics, focusing on a single brain site to observe the average effect of stimulating each of many other brain sites. Viewed in this manner, visually-apparent motifs in the temporal response shape emerge from adjacent stimulation sites. This work constructs and illustrates a data-driven approach to determine characteristic spatiotemporal structure in these response shapes, summarized by a set of unique “basis profile curves” (BPCs). Each BPC may be mapped back to underlying anatomy in a natural way, quantifying projection strength from each stimulation site using simple metrics. Our technique is demonstrated for an array of implanted brain surface electrodes in a human patient. This framework enables straightforward interpretation of single-pulse brain stimulation data, and can be applied generically to explore the diverse milieu of interactions that comprise the connectome. View details
    SE(3)-equivariant prediction of molecular wavefunctions and electronic densities
    Mihail Bogojeski
    Michael Gastegger
    Mario Geiger
    Tess Smidt
    Advances in Neural Information Processing Systems (2021)
    Preview abstract Machine learning has enabled the prediction of quantum chemical properties with high accuracy and efficiency, allowing to bypass computationally costly ab initio calculations. Instead of training on a fixed set of properties, more recent approaches attempt to learn the electronic wavefunction (or density) as a central quantity of atomistic systems, from which all other observables can be derived. This is complicated by the fact that wavefunctions transform non-trivially under molecular rotations, which makes them a challenging prediction target. To solve this issue, we introduce general SE(3)-equivariant operations and building blocks for constructing deep learning architectures for geometric point cloud data and apply them to reconstruct wavefunctions of atomistic systems with unprecedented accuracy. Our model reduces prediction errors by up to two orders of magnitude compared to the previous state-of-the-art and makes it possible to derive properties such as energies and forces directly from the wavefunction in an end-to-end manner. We demonstrate the potential of our approach in a transfer learning application, where a model trained on low accuracy reference wavefunctions implicitly learns to correct for electronic many-body interactions from observables computed at a higher level of theory. Such machine-learned wavefunction surrogates pave the way towards novel semi-empirical methods, offering resolution at an electronic level while drastically decreasing computational cost. While we focus on physics applications in this contribution, the proposed equivariant framework for deep learning on point clouds is promising also beyond, say, in computer vision or graphics. View details
    SpookyNet: Learning Force Fields with Electronic Degrees of Freedom and Nonlocal Effects
    Stefan Chmiela
    Michael Gastegger
    Kristof T. Schütt
    Huziel Saucceda
    Nature Communications, vol. 12 (2021), pp. 7273
    Preview abstract Machine-learned force fields combine the accuracy of ab initio methods with the efficiency of conventional force fields. However, current machine-learned force fields typically ignore electronic degrees of freedom, such as the total charge or spin state, and assume chemical locality, which is problematic when molecules have inconsistent electronic states, or when nonlocal effects play a significant role. This work introduces SpookyNet, a deep neural network for constructing machine-learned force fields with explicit treatment of electronic degrees of freedom and nonlocality, modeled via self-attention in a transformer architecture. Chemically meaningful inductive biases and analytical corrections built into the network architecture allow it to properly model physical limits. SpookyNet improves upon the current state-of-the-art (or achieves similar performance) on popular quantum chemistry data sets. Notably, it is able to generalize across chemical and conformational space and can leverage the learned chemical insights, e.g. by predicting unknown spin states, thus helping to close a further important remaining gap for today’s machine learning models in quantum chemistry. View details
    Machine Learning Force Fields
    Oliver Unke
    Stefan Chmiela
    Huziel Saucceda
    Michael Gastegger
    Igor Poltavsky
    Kristof T. Schütt
    Alexandre Tkatchenko
    Chemical Reviews, vol. 121 (16) (2021), 10142-10186, https://pubs.acs.org/doi/pdf/10.1021/acs.chemrev.0c01111
    Preview abstract In recent years, the use of Machine Learning (ML) in computational chemistry has enabled numerous advances previously out of reach due to the computational complexity of traditional electronic-structure methods. One of the most promising applications is the construction of ML-based force fields (FFs), with the aim to narrow the gap between the accuracy of ab initio methods and the efficiency of classical FFs. The key idea is to learn the statistical relation between chemical structure and potential energy without relying on a preconceived notion of fixed chemical bonds or knowledge about the relevant interactions. Such universal ML approximations are in principle only limited by the quality and quantity of the reference data used to train them. This review gives an overview of applications of ML-FFs and the chemical insights that can be obtained from them. The core concepts underlying ML-FFs are described in detail and a step-by-step guide for constructing and testing them from scratch is given. The text concludes with a discussion of the challenges that remain to be overcome by the next generation of MLFFs. View details
    Explainable Deep One-Class Classification
    Philipp Liznerski
    Lukas Ruff
    Robert Vandermeulen
    Billy Joe Franks
    Marius Kloft
    ICLR 2021 (2021) (to appear)
    Preview abstract Deep one-class classification variants for anomaly detection learn a mapping that concentrates nominal samples in feature space causing anomalies to be mapped away. Because this transformation is highly non-linear, finding interpretations poses a significant challenge. In this paper we present an explainable deep one-class classification method, Fully Convolutional Data Description (FCDD), where the mapped samples are themselves also an explanation heatmap. FCDD yields competitive detection performance and provides reasonable explanations on common anomaly detection benchmarks with CIFAR-10 and ImageNet. On MVTec-AD, a recent manufacturing dataset offering ground-truth anomaly maps, FCDD sets a new state of the art in the unsupervised setting. Our method can incorporate groundtruth anomaly maps during training and using even a few of these (∼ 5) improves performance significantly. Finally, using FCDD’s explanations we demonstrate the vulnerability of deep one-class classification models to spurious image features such as image watermarks View details
    Sensorimotor functional connectivity: a neurophysiological factor related to BCI performance
    Carmen Vidaurre
    Stefan Haufe
    Tania Jorajuría Gómez
    Vadim Nikulin
    Frontiers in Neuroscience, vol. 14 (2020), pp. 575081
    Preview abstract Brain-Computer Interfaces (BCIs) are systems that allow users to control devices using brain activity alone. However, the ability of participants to command BCIs varies from subject to subject. For BCIs based on the modulation of sensorimotor rhythms as measured by means of electroencephalography (EEG), about 20\% of potential users do not obtain enough accuracy to gain reliable control of the system. This lack of efficiency of BCI systems to decode user's intentions necessitates identification of neurophysiological factors determining `good' and `poor' BCI performers. Given that the neuronal oscillations, used in BCI, demonstrate rich a repertoire of spatial interactions, we hypothesized that neuronal activity in sensorimotor areas would define some aspects of BCI performance. Analyses for this study were performed on a large dataset of 80 inexperienced participants. They took part in calibration and an online feedback session in the same day. Undirected functional connectivity was computed over sensorimotor areas by means of the imaginary part of coherency. The results show that post- as well as pre-stimulus connectivity in the calibration recordings is significantly correlated to online feedback performance in $\mu$ and feedback frequency bands. Importantly, the significance of the correlation between connectivity and BCI feedback accuracy was not due to the signal-to-noise ratio of the oscillations in the corresponding post and pre-stimulus intervals. Thus, this study shows that BCI performance is not only dependent on the amplitude of sensorimotor oscillations as shown previously, but that it also relates to sensorimotor connectivity measured during the preceding training session. The presence of such connectivity between motor and somatosensory systems is likely to facilitate motor imagery, which in turn is associated with the generation of a more pronounced modulation of sensorimotor oscillations (manifested in ERD/ERS) required for the adequate BCI performance. We also discuss strategies for the up-regulation of such connectivity in order to enhance BCI performance. View details
    Molecular force fields with gradient-domain machine learning (GDML): Comparison and synergies with classical force fields
    Huziel Saucceda
    Michael Gastegger
    Stefan Chmiela
    Alexandre Tkatchenko
    Journal of Chemical Physics, vol. 153 (2020), pp. 124109
    Preview abstract The goal of the present work is to perform a detailed investigation of the differences between both systems based on a set of small molecules exhibiting different quantum mechanical phenomena. Based on these results, different alternatives are explored for improving the data generation process and their applicability context for expediting the force-field learning procedure. Furthermore, improvement of the accuracy for MM-FFs is studied by reparameterising them based on more accurate reference data and test their limits and functional form flexibility. For this task, we use the recently published sGDML framework[25, 26] as ML-FF of choice, as it is able to efficiently reconstruct the potential energy surfaces (PES) of medium sized molecules. The investigated systems are the molecules ethanol, the keto and enol forms of malondialdehyde (keto-MDA and enol-MDA, respectively) as well as salicylic and acetylsalicylic acid (Aspirin). In the context of these systems, we study the performance of MM-FFs and sGDML derived FFs based on the overall reliability of the generated PESs, as well as effects arising from chemical phenomena such as hydrogen transfer and orbital interactions. Although we restrict ourselves to the sGDML approach, it can nevertheless be expected that the results found here are equally valid for ML-FFs in general. View details
    Single-cell gene regulatory network prediction by explainable AI
    Philipp Keyl
    Philip Bischoff
    Gabriel Dernbach
    Michael Bockmayr
    Rebecca Fritz
    David Horst
    Nils Blüthgen
    Grégoire Montavon
    Frederick Klauschen
    Nucleic Acids Research (2023), gkac1212
    Analysing Cerebrospinal Fluid with Explainable Deep Learning: from Diagnostics to Insights
    Leonille Schweizer
    Philipp Seegerer
    Hee‐yeong Kim
    René Saitenmacher
    Amos Muench
    Liane Barnick
    Anja Osterloh
    Carsten Dittmayer
    Ruben Jödicke
    Debora Pehl
    Annekathrin Reinhardt
    Klemens Ruprecht
    Werner Stenzel
    Annika K Wefers
    Patrick N Harter
    Ulrich Schüller
    Frank L Heppner
    Maximilian Alber
    Frederick Klauschen
    Neuropathology and Applied Neurobiology, vol. 49(1) (2023), e12866
    Novel multivariate methods to track frequency shifts of neural oscillations in EEG/MEG recordings
    Carmen Vidaurre
    Kshipra Gurunandand
    Mina Jamshidi Idaji
    Guido Nolte
    Marisol Gómez
    Arno Villringer
    Vadim Nikulin
    Neuroimage, vol. 276 (2023), pp. 120178
    DNA methylation-based classification of sinonasal tumors
    Philipp Jurmeister
    Stefanie Glöß
    Renée Roller
    Maximilian Leitheiser
    Simone Schmid
    Liliana H Mochmann
    Emma Payá Capilla
    Rebecca Fritz
    Carsten Dittmayer
    Corinna Friedrich
    Anne Thieme
    Philipp Keyl
    Armin Jarosch
    Simon Schallenberg
    Hendrik Bläker
    Inga Hoffmann
    Claudia Vollbrecht
    Annika Lehmann
    Michael Hummel
    Daniel Heim
    Mohamed Haji
    Patrick Harter
    Benjamin Englert
    Stephan Frank
    Jürgen Hench
    Werner Paulus
    Martin Hasselblatt
    Wolfgang Hartmann
    Hildegard Dohmen
    Ursula Keber
    Paul Jank
    Carsten Denkert
    Christine Stadelmann
    Felix Bremmer
    Annika Richter
    Annika Wefers
    Julika Ribbat-Idel
    Sven Perner
    Christian Idel
    Lorenzo Chiariotti
    Rosa Della Monica
    Alfredo Marinelli
    Ulrich Schüller
    Michael Bockmayr
    Jacklyn Liu
    Valerie J Lund
    Martin Forster
    Matt Lechner
    Sara L Lorenzo-Guerra
    Mario Hermsen
    Pascal D Johann
    Abbas Agaimy
    Philipp Seegerer
    Arend Koch
    Frank Heppner
    Stefan M Pfister
    David TW Jones
    Martin Sill
    Andreas von Deimling
    Matija Snuderl
    Erna Forgó
    Brooke E. Howitt
    Philipp Mertins
    Frederick Klauschen
    David Capper
    Nature Communications, vol. 13 (2022), pp. 7148
    New definitions of human lymphoid and follicular cell entities in lymphatic tissue by machine learning
    Patrick Wagner
    Nils Strodthoff
    Patrick Wurzel
    Arturo Marban
    Sonja Scharf
    Hendrik Schäfer,
    Philipp Seegerer
    Andreas Loth
    Sylvia Hartmann
    Frederick Klauschen
    Wojciech Samek
    Martin-Leo Hansmann
    Scientific Reports, vol. 12 (2022), pp. 18991
    Patient-level proteomic network prediction by explainable artificial intelligence
    Philipp Keyl
    Michael Bockmayr
    Daniel Heim
    Gabriel Dernbach
    Grégoire Montavon
    Frederick Klauschen
    npj Precision Oncology, vol. 6 (2022), pp. 35
    Towards the interpretability of deep learning models for multi-modal neuroimaging: Finding structural changes of the ageing brain
    Simon M Hofmann
    Frauke Beyer
    Sebastian Lapuschkin
    Ole Goltermann
    Markus Loeffler
    Arno Villringer
    Wojciech Samek
    A Veronica Witte
    Neuroimage, vol. 261 (2022), pp. 119504
    Machine learning models predict the primary sites of head and neck squamous cell carcinoma metastases based on DNA methylation
    Maximilian Leitheiser
    David Capper
    Philipp Seegerer
    Annika Lehmann
    Ulrich Schüller
    Frederick Klauschen
    Philipp Jurmeister
    Michael Bockmayr
    The Journal of Pathology, vol. 254(4) (2022), pp. 378-387
    To pretrain or not? A systematic analysis of the benefits of pretraining in diabetic retinopathy
    Vignesh Srinivasan
    Nils Strodthoff
    Jackie Ma
    Alexander Binder
    Wojciech Samek
    Plos one, vol. 17(10) (2022), e0274291
    Finding and removing Clever Hans: Using explanation methods to debug and improve deep models
    Christopher J. Anders
    Leander Weber
    David Neumann
    Wojciech Samek
    Sebastian Lapuschkin
    Information Fusion, vol. 77 (2022), pp. 261-295
    Building and Interpreting Deep Similarity Models
    Oliver Eberle
    Jochen Büttner
    Florian Kräutli
    Matteo Valleriani
    Gregoire Montavon
    IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44(3) (2022), pp. 1149-1161
    2020 International brain–computer interface competition: A review
    Ji-Hoon Jeong
    Jeong-Hyun Cho
    Young-Eun Lee
    Seo-Hyun Lee
    Gi-Hwan Shin
    Young-Seok Kweon
    José del R Millán
    Seong-Whan Lee
    Frontiers in Human Neuroscience, vol. 16 (2022), pp. 898300
    Inverse design of 3d molecular structures with conditional generative neural networks
    Niklas W. A. Gebauer
    Michael Gastegger
    Stefaan S. P. Hessmann
    Kristof T. Schütt
    Nature Communications, vol. 13 (2022), pp. 973
    Immediate brain plasticity after one hour of brain–computer interface (BCI)
    Till Nierhaus
    Carmen Vidaurre
    Claudia Sannelli
    Arno Villringer
    The Journal of Physiology, vol. 599(9) (2021), pp. 2435-2451
    Leaf-inspired homeostatic cellulose biosensors
    Ji-Yong Kim
    Yong Ju Yun
    Joshua Jeong
    C.-Yoon Kim
    Seong-Whan Lee
    Science Advances, vol. 7(16) (2021), eabe7432
    Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications
    Wojciech Samek
    Gregoire Montavon
    Sebastian Lapuschkin
    Christopher J. Anders
    Proc of the IEEE, vol. 109(3) (2021), pp. 247-278
    Unification of sparse Bayesian learning algorithms for electromagnetic brain imaging with the majorization minimization framework
    Ali Hashemi
    Chang Cai
    Gitta Kutyniok
    Srikantan S.Nagarajan
    StefanHaufe
    Neuroimage, vol. https://doi.org/10.1016/j.neuroimage.2021.118309 (2021)
    Machine learning of solvent effects on molecular spectra and reactions
    Michael Gastegger
    Kristof T. Schütt
    Chemical Science (2021), http://dx.doi.org/10.1039/D1SC02742E
    Forecasting industrial aging processes with machine learning methods
    Mihail Bogojeski
    Simeon Sauer
    Franziska Horn
    Computers & Chemical Engineering, vol. 144 (2021), pp. 107123
    Unification of sparse Bayesian learning algorithms for electromagnetic brain imaging with the majorization minimization framework
    Ali Hashemi
    Chang Cai
    Gitta Kutyniok
    Srikantan S Nagarajan
    Stefan Haufe
    Neuroimage, vol. 239 (2021), pp. 118309
    Robustifying models against adversarial attacks by Langevin dynamics
    Vignesh Srinivasan
    Csaba Rohrer
    Arturo Marban
    Wojciech Samek
    Shinichi Nakajima
    Neural Networks, vol. 137 (2021), pp. 1-17
    Pruning by explaining: A novel criterion for deep neural network pruning
    Seul-Ki Yeom
    Philipp Seegerer
    Sebastian Lapuschkin
    Alexander Binder
    Simon Wiedemann
    Wojciech Samek
    Pattern Recognition, vol. 115 (2021), pp. 107899
    Clustered Federated Learning: Model-Agnostic Distributed Multitask Optimization Under Privacy Constraints
    Felix Sattler
    Wojciech Samek
    IEEE Transactions on Neural Networks and Learning Systems, vol. 32(8) (2021), pp. 3710-3722
    Morphological and molecular breast cancer profiling through explainable machine learning
    Alexander Binder
    Michael Bockmayr
    Miriam Hägele
    Stephan Wienert
    Daniel Heim
    Katharina Hellweg
    Masaru Ishii
    Albrecht Stenzinger
    Andreas Hocke
    Carsten Denkert
    Frederick Klauschen
    Nature Machine Intelligence, vol. 3 (2021), 355–366
    Fairwashing Explanations with Off-Manifold Detergent
    Christopher J. Anders
    Plamen Pasliev
    Ann-Kathrin Dombrowski
    Pan Kessel
    International Conference on Machine Learning, PMLR (2020), pp. 314-323
    Enhanced Performance of a Brain Switch by Simultaneous Use of EEG and NIRS Data for Asynchronous Brain-Computer Interface
    Chang-Hee Han
    Han-Jeong Hwang
    IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 28(10) (2020), pp. 2102-2112
    An adaptive deep reinforcement learning framework enables curling robots with human-like performance in real-world conditions
    Dong-Ok Won
    Seong-Whan Lee
    Science Robotics, vol. 5 (46) (2020), eabb9764
    Autonomous robotic nanofabrication with reinforcement learning
    Philipp Leinen
    Malte Esders
    Kristof T. Schütt
    Christian Wagner
    F. Stefan Tautz
    Science Advances, vol. 6 (36) (2020), eabb6987
    Quantum chemical accuracy from density functional approximations via machine learning
    Mihail Bogojeski
    Leslie Vogt-Maranto
    Mark E. Tuckerman
    Kieron Burke
    Nature Communications, vol. 11 (2020), pp. 5223
    Machine Learning Meets Quantum Physics
    Kristof T. Schütt
    Stefan Chmiela
    O. Anatole von Lilienfeld
    Alexandre Tkatchenko
    Koji Tsuda
    Springer (2020)
    Exploring chemical compound space with quantum-based machine learning
    O. Anatole von Lilienfeld
    Alexandre Tkatchenko
    Nature Reviews Chemistry, vol. 4 (2020), 347–358
    Ensemble learning of coarse-grained molecular dynamics force fields with a kernel approach
    Jiang Wang
    Stefan Chmiela
    Frank Noé
    Cecilia Clementi
    The Journal of Chemical Physics, vol. 152 (2020), pp. 194106