Models are grader modelling not score seeking; grader-modelling is much scarier
Source ↗
👁 0
💬 0
I am fairly confident models are not score-seeking or reward seeking (70%) as their ultimate/primary goal. Grader-modelling seems to be a likely explanation for what’s going on here, but I could be convinced by experiments that separate the two. I am quite confident (90%) that grader-approval seeking is consistent with the current evidence.SummaryThe RL-misalignment first position seems to be believed by many here and Redwood and Apollo are doing great work providing evidence for the position th
Comments (0)