The J-lens intermediate swap acts partly through the answer directions
Source ↗
👁 0
💬 0
TL;DRWhen a language model answers a multi-hop prompt, does it first compute the intermediate concept internally, or does it jump straight to the answer through some shortcut? The global workspace paper investigates this question using the Jacobian lens, which maps activations at a given layer to the tokens that they are poised to verbalize, with the activations possessing a coordinate in each token’s lens direction. The authors intervene on the model through an intermediate swap that exchanges
Comments (0)