CarrotParsnipBench: A Lightweight Multi-Agent Deception and Detection Eval
Source ↗
👁 0
💬 0
TL:DR Earlier this year I introduced the very simple Social Deduction game of Carrot-Parsnip and showed that most models are better-than-random at identifying Parsnips (the hidden, deceptive role) and that being a strong Parsnip (skilled at strategic deception) does not perfectly correlate with being a strong Carrot (skilled at deception detection).I have recently completed a substantially larger experiment with 10 models and over 5000 games. I have attempted to fit the results to a four-paramet
Comments (0)