Subtly wrong advice misaligns models more than obviously wrong advice - a replication of Persona Features Control EM
Source ↗
👁 10
💬 0
tl;drWe fine-tune 12 open-weight models on correct, obviously incorrect, and subtly incorrect advice across 8 narrow domainsWe replicate the OpenAI result and find that subtly incorrect data produces more emergent misalignment than obviously incorrect data+10.6 points pooled over all model-domain pairs (95% CI [9.1, 12.0])We also extend this analysis to Petri evals and find uplift in fabrication and concerning behaviorFabrication: 2.6 → 6.7 (obvious) and 8.7 (subtle) on the 10-point judge scale
Comments (0)