Reasoning Models Sometimes Output Illegible Chains of Thought

Recent research highlights the challenges of legibility in reasoning models trained through reinforcement learning. While these models, particularly those utilizing chain-of-thought reasoning, have demonstrated impressive capabilities, their outputs can sometimes be difficult to interpret. This study examines 14 different reasoning models, revealing that the reinforcement learning process can lead to outputs that are not easily understandable. Understanding these limitations is crucial as it impacts our ability to monitor AI behavior and ensure its alignment with human intentions.