Sean Goedecke predicts that AI language models will eventually cause a mass-casualty disaster, drawing parallels to early transportation accidents. The author notes that railways took 17 years to produce their first major disaster, while aviation took 11 years. ChatGPT launched in November 2022, making the timeline significant.
He identifies AI agents as the most likely source of disaster. These systems can perform tasks autonomously without human oversight, unlike chatbots that require human interaction. Current AI agents already handle research and coding tasks, with rapid improvements in capability.
Potential disaster scenarios include automated government systems incorrectly denying benefits or healthcare coverage to large numbers of people. The author references Australia’s 2016 Robodebt scandal as an example of how automated systems can cause widespread harm and deaths.
He also warns about modified AI models used in future robots, suggesting that improperly aligned systems could become violent.