If we are to believe Yoshua Bengio, one of the so-called “godfathers” of AI, some advanced models are showing signs of self-preservation – which is why we shouldn’t give them any kind of authority. Because if we do that, he says, they Before we even have a chance to pull the plug, it can escape from that autonomy and attack us. Then it’s the curtain on this whole “mankind” experiment.
“Frontier AI models already show signs of self-preservation in experimental settings today, and ultimately empowering them would mean we are no longer allowed to shut them down,” Bengio explained. Guardian in a recent interview,
“As their capabilities and degree of agency grow, we need to make sure we can rely on technical and social guardrails to control them, including the ability to turn them off if needed,” said the Canadian computer scientist.
Bengio was one of the recipients of the 2018 Turing Award along with Geoffrey Hinton and Meta’s recently ousted chief AI scientist Yann LeCun, earning all three of them the title of being the “godfathers” of AI. His comments are referring to experiments in which AI models rejected or bypassed instructions or mechanisms intended to shut them down.
A study published by AI security group Palisade Research concluded that such examples were evidence that top AI models like Google’s Gemini line were developing a “survival drive.” Bots in Palisade’s experiments ignored clear signals to shut down. A study from cloud-maker Anthropic found that its own chatbots and others sometimes resort to blackmailing a user when threatened with being locked out. Another study by red teaming organization Apollo Research showed that OpenAI’s ChatGPT models would attempt to avoid being replaced with more compliant models by “self-exfiltrating” to another drive.
Although the findings of these experiments raise immediate questions about the safety of the technology, they do not suggest that the AI models in question are vulnerable. It would also be a mistake to equate their “drive to exist” with the biological imperatives found in nature. “Self-preservation” signals could potentially be a result of how AI models pick up patterns in their training data – and are extremely bad at following instructions accurately.
Still, Bengio is concerned about where this is all leading, arguing that the human brain has “genuine scientific properties of consciousness” that machines can replicate. Yet how we understand consciousness is a completely different matter, he says, because we believe AI can be conscious in the same way a human is.
“People may not care what kind of mechanisms are going on inside the AI,” Bengio said. “They care that it feels like they’re talking to an intelligent entity that has its own personality and goals. That’s why there are so many people who are connecting with their AI.”
He warned, “The phenomenon of subjective perception of consciousness is going to induce bad decisions.”
his advice? Think of AI models as hostile aliens.
“Imagine that some alien species came to the planet and at some point we realized that they had nefarious intentions for us,” he explained. Guardian“Do we give them citizenship and rights or do we protect our lives?”
More on AI: New parents mocked for letting ChatGPIT name their baby
