Eval Awareness vs. Eval Paranoia
Source ↗
👁 0
💬 0
I have said several times that I think frontier models are not that eval aware. I think this is an important fact to unlearn about current models, in order to interpret their current behavior. Others have found this idea quite curious, and asked me for concrete examples. I'm going to try to explain why I believe what I believe and attempt to give a tentative explanation for the behavior that I do see, which I think people sometimes mistake for consistent or understandable evaluation awareness. I
Comments (0)