Sjoerd van Steenkiste
Sjoerd is a research scientist at Google Research. His current research focus is twofold: (1) Analyzing LLMs through the lens of human cognition and improving their reasoning capabilities; and (2) Approaches to learning representation of 4D scenes that capture meaningful structure (objects, geometry, etc.). More broadly, he is interested in multimodality (eg. combining vision + language), compositional generalization, learning structured 'symbol-like' representations with neural networks, and the binding problem.
Before joining Google he was a Postdoc at the Dalle Molle Institute for Artificial Intelligence (IDSIA) in the Italian-speaking part of Switzerland with Jürgen Schmidhuber, which is also where he completed his PhD in 2020. Prior to joining Google as a Research Scientist he was an intern with the Google Brain team in Zurich in 2018.
Research Areas
Authored Publications
Sort By
Google
DreamSync: Aligning Text-to-Image Generation with Image Understanding Models
Jiao Sun
Yushi Hu
Deqing Fu
Royi Rassin
Su Wang
Charles Herrmann
Ranjay Krishna
Synthetic Data for Computer Vision Workshop @ CVPR 2024
DORSal: Diffusion for Object-centric Representations of Scenes et al.
Allan Jabri
Emiel Hoogeboom
Thomas Kipf
International Conference on Learning Representations (2024)
Test-time Adaptation with Slot-centric Models
Mihir Prabhudesai
Anirudh Goyal
Gaurav Aggarwal
Thomas Kipf
Deepak Pathak
Katerina Fragkiadaki
International Conference on Machine Learning (2023), pp. 28151-28166
Scaling Vision Transformers to 22 Billion Parameters
Josip Djolonga
Basil Mustafa
Piotr Padlewski
Justin Gilmer
Mathilde Caron
Rodolphe Jenatton
Lucas Beyer
Michael Tschannen
Anurag Arnab
Carlos Riquelme
Matthias Minderer
Gamaleldin Elsayed
Fisher Yu
Avital Oliver
Fantine Huot
Mark Collier
Vighnesh Birodkar
Yi Tay
Alexander Kolesnikov
Filip Pavetić
Thomas Kipf
Xiaohua Zhai
Neil Houlsby
Arxiv (2023)
Unsupervised Learning of Temporal Abstractions with Slot-based Transformers
Anand Gopalakrishnan
Jürgen Schmidhuber
Kazuki Irie
Neural Computation, 35 (2023), pp. 593-626
Invariant Slot Attention: Object Discovery with Slot-Centric Reference Frames
Ondrej Biza
Gamaleldin Elsayed
Thomas Kipf
International Conference on Machine Learning (2023), pp. 2507-2527
Object Scene Representation Transformer
Filip Pavetić
Leonidas Guibas
Klaus Greff
Thomas Kipf
Advances in Neural Information Processing Systems (2022), pp. 9512-9524
SAVi++: Towards End-to-End Object-Centric Learning from Real-World Videos
Gamaleldin Fathy Elsayed
Klaus Greff
Michael Mozer
Thomas Kipf
Advances in Neural Information Processing Systems (2022), pp. 28940-28954
Exploring through Random Curiosity with General Value Functions
Aditya Ramesh
Louis Kirsch
Jürgen Schmidhuber
Advances in Neural Information Processing Systems (2022), pp. 18733-18748
Investigating object compositionality in Generative Adversarial Networks
Karol Kurach
Jürgen Schmidhuber
Sylvain Gelly
Neural Networks, 130 (2020), pp. 309-325
