The Unintelligibility is Ours: Notes on Chain-of-Thought
Source ↗
👁 0
💬 0
Many people seem to think that the chains-of-thought in RL-trained LLMs are under a great deal of "pressure" to cease being English. The idea is that, as LLMs solve harder and harder problems, they will eventually slide into inventing a "new language" that lets them solve problems better, more efficiently, and in fewer tokens, than thinking in a human-intelligible chain-of-thought.
I'm less sure this will happen, or that it will happen before some kind of ASI. As a high-level intuition pump for
I'm less sure this will happen, or that it will happen before some kind of ASI. As a high-level intuition pump for
Comments (0)