Score-based Causal Representation Learning: Linear and General Transformations

This paper addresses intervention-based causal representation learning (CRL) under a
general nonparametric latent causal model and an unknown transformation that maps the
latent variables to the observed variables. Linear and general transformations are investigated. The paper addresses both the identifiability and achievability aspects. Identifiability
refers to determining algorithm-agnostic conditions that ensure the recovery of the true
latent causal variables and the underlying latent causal graph. Achievability refers to the algorithmic aspects and addresses designing algorithms that achieve identifiability guarantees.
By drawing novel connections between score functions (i.e., the gradients of the logarithm of
density functions) and CRL, this paper designs a score-based class of algorithms that ensures
both identifiability and achievability. First, the paper focuses on linear transformations and
shows that one stochastic hard intervention per node suffices to guarantee identifiability. It
also provides partial identifiability guarantees for soft interventions, including identifiability
up to mixing with parents for general causal models and perfect recovery of the latent graph
for sufficiently nonlinear causal models. Secondly, it focuses on general transformations
and demonstrates that two stochastic hard interventions per node are sufficient for identifiability. This is achieved by defining a differentiable loss function whose global optima
ensure identifiability for general CRL. Notably, one does not need to know which pair of
interventional environments has the same node intervened. Finally, the theoretical results
are empirically validated via experiments on structured synthetic data and image data.