Deductive Logic in Language Models: Horizontal vs. Vertical Reasoning

Today's article comes from the journal of Machine Learning and Knowledge Extraction. The authors are Maltoni et al., from the University of Bologna, in Italy. In this paper, They want to know two things: How a transformer actually implements multi-step deductive reasoning internally. And whether that computation differs when the reasoning happens through generated intermediate tokens versus through the model's layers.