Linear Probes add little for Verifiable Reward Hacking
Source ↗
👁 0
💬 0
SummaryTested whether linear probes can detect reward hacking early during GRPO training on a small model.Used a synthetic arithmetic task with a planted bug in the reward checker.Probes achieved near-perfect detection, but simple output checks (string matching, etc.) worked just as well.The model only learned obvious “lazy” hacks, more subtle ones never appeared.Conclusion: In verifiable reward settings, probes offer little advantage over checking the output directly.IntroductionI wanted to see
Comments (0)