opinion tech single source: The Verge

The Brake Pedal Analogy: Is Current AI Safety Fundamentally Broken?

Microsoft issued a thirty-seven page document outlining its Humanist AI Code of Conduct.

Mustafa Suleyman, CEO of Microsoft AI, laid out a sobering argument this week, suggesting that the prevailing methods for ensuring AI safety might have reached a breaking point. In conjunction with releasing a substantial 37-page "Humanist AI Code of Conduct," Suleyman offered a pointed critique of approaches taken by competitors like Anthropic regarding concepts such as model welfare.

He paints a picture of staggering advancement. Comparing early models to what is coming next—GPT-9, for instance—represents leaps involving three orders of magnitude more compute and thousands of times more processing power. The resulting capabilities will be breathtakingly immense.

This rapid climb forces him to employ a harsh analogy: if one were driving a car whose brakes failed ten percent of the time, the solution wouldn't be tweaking the pedal; the entire braking technology would be considered flawed. He applies this exact logic to alignment efforts in AI.

This raises the central question: Can our current path to making models completely safe succeed? Suleyman insists against simplistic binaries—there is no simple answer among stopping everything, accelerating blindly, relying solely on regulation, or trusting only industry self-governance. All those positions are incomplete.

His stance is pragmatic yet urgent: if alignment fails to solve the problem, the industry must halt development entirely until a viable new technological safeguard emerges. Yet, he anchors this caution in acknowledging that alignment has shown promise throughout recent progress.

However, he stresses that focusing only on incremental fixes misses larger dangers inherent in autonomous systems—specifically recursive self-improvement and coordinated agency—issues that serious thinkers have debated since 2017. Patience is needed alongside urgency as the field navigates this massive inflection point.