Arun Kandoor
Research Areas
Authored Publications
Sort By
Preview abstract
Large language models (LLMs) have demonstrated remarkable performance in tasks that require reasoning abilities. Motivated by recent works showing evidence of LLMs being able to plan and reason on abstract reasoning problems in context, we conduct a set of controlled experiments on a synthetic propositional logic problem to provide a mechanistic understanding of how such abilities arise. In particular, for a decoder-only transformer trained solely on our synthetic dataset,
we identify the specific mechanisms by which a three-layer Transformer solves the reasoning task. In particular, we identify certain ``planning'' and reasoning circuits which require cooperation between the attention blocks to in totality implement the desired reasoning algorithm.
We also find that deeper models with greater number of attention heads exhibit a stronger performance on solving more complex variants of our logic problem.
View details
