Humanity isn’t ready for the coming intelligence explosion

SOCIETY dictates that the acceptable risk of a catastrophic meltdown for a nuclear power plant is roughly one in a million. Experts in artificial intelligence estimate the risk of an AI-caused catastrophic event at 10 to 50 per cent.

Strikingly, this concern is being openly voiced by the very people who have the strongest incentives to project confidence rather than alarm: the founders of the largest AI laboratories.

AI leaders are in a race they feel unable to escape. AI investments are set to outspend the Manhattan Project a hundredfold, even adjusting for inflation. Yet, spending on AI safety might be 100 times less.