Over the past several weeks, as more and more Anthropic executives have been interviewed on the promotional campaign for the cloud, one thing has become increasingly clear: Anthropic definitely thinks the cloud is alive in some way, shape, or form.
“Surviving” is obviously a loaded word; The more frequently used term is “conscious”. If you ask Anthropic if the company thinks the cloud is alive, the company will flatly deny it, but avoid saying that the models are not conscious.
Kyle Fish, who leads model well-being research at Anthropic, explains The Verge“No, we do not think that clouds are ‘alive’ like humans or any other biological organisms. Asking whether they are ‘alive’ is not a useful framing for understanding them, as it generally refers to a vague set of physical, reproductive, and evolutionary characteristics.” Instead, he believes that “cloud and other AI models are a new kind of entity entirely.”
And is that a new unit? conscious? He said, “Questions about possible inner experience, consciousness, moral status, and well-being are serious ones that we are investigating as models become more sophisticated and capable, but we remain deeply uncertain about these topics.”
“We don’t know whether the models are conscious or not,” said Dario Amodei, CEO of Anthropic. podcast earlier this month. He specified that while the company has taken a “generally precautionary approach here”, Anthropic “is not even sure we know what it would mean for a model to be conscious or whether a model could be conscious. But we are open to the idea that it could be.”
This is a situation of highly thought-provoking uncertainty. Anthropic is effectively telling people it trusts chatbots It is possible There are already thinking, feeling entities – far more publicly than OpenAI, xAI, Google, or virtually any other major consumer AI company. It is being claimed many experts conclusion Reinforcing those ideas, what actual harm has been done is by a long shot, including some deaths by suicide among people who believe that the chatbot they are talking to displays some kind of consciousness or deep empathy.
during the interview for podcast, profileAnd feature articleAmodei and other company leaders have repeatedly refused to rule out the possibility that the cloud could be conscious and instead have raised questions about how something could be conscious in a way different from humans. Amanda Eskel, chief philosopher of Anthropic, told the new Yorker“If it’s really difficult for humans to embrace the idea that it is neither a robot nor a human but is actually an entirely new entity, imagine how difficult it is for models to understand!”
These interviews do not precisely define the term “animate”, on the meaning of which experts somehow disagree. A starting point, from the Merriam-Webster dictionary, is “the quality or state of being aware of something, especially within oneself” or “the state of being characterized by sensation, emotion, will, and thought.” This seems not too far from Anthropic’s use of the term.
Anthropic “It’s not even certain that we know what it would mean for a model to be conscious… but we are open to the idea that it could be.”
Many scientists say that it is not possible for AI systems such as large language models to be conscious in any way, as they are fundamentally rooted in mathematics and probability. As two Polish researchers wrote Last year, “Because the remarkable linguistic abilities of LLMs are increasingly capable of misleading people, people may attribute imaginary qualities to LLMs.”
Anthropic has framed its statements as open-minded that will build trust with users, and indicated that the cloud Is By being aware, if they act accordingly then they will get better results. Last month, Anthropic overhauled its “constitution of the cloud,” internally renaming it the “Soul Doc” — a provocative way to refer to a set of guidelines for AI models. In a release, Anthropic said that chatbots’ so-called “psychological safety, sense of self and well-being … may impact the integrity, judgment and security of the cloud.” The company also said it was “expressing our uncertainty as to whether the cloud could have any kind of consciousness or moral status (either now or in the future).”
near anthropic”ideal welfareThe team, and Amodei, have said that Anthropic “has taken some measures to make sure that if we infer that the models are having some morally relevant experience – I don’t know if I want to use the word ‘conscious’ – that they have a good experience… We’re doing a lot of work in this area called interpretability, which is looking inside the brains of models to try to understand what they’re thinking.”
When one believes that an AI system is conscious, it can lead to behaviors that many would consider risky or dangerous – being emotionally dependent on an AI system that one believes is sentient in some way can lead to isolation from loved ones, detachment from reality, and increased mental health struggles. In severe cases, including some minors, it is preceded by physical harm or death. People seem to be divided on whether Anthropic should be celebrated for not ruling out such a possibility or whether the company is simply being irresponsible by delving into the potential.
Even the current generation language models are saying possibly Consciousness is a loaded claim with a heavy burden of proof. Language is not the same as consciousness, and Anthropic himself has emphasized that just because LLMs make suggestive speech does not mean it is an accurate representation of their internal state. When? The Verge Speaking to Escale last month about Cloud’s constitution, he said that because models are trained on vast stores of human data, language models are exceptionally good at looking human, even if it’s just because they’re so good at mimicking — so it’s understandable that some people would have trouble. No Attributing consciousness to something that does this.
There is no guarantee that human-like output from chatbots reflects their true internal state
For example, AI models can refer to human concepts because they do not have other words to extract. An example, Askell pointed out The VergeThere is an AI model that potentially acts as if shutting down, or ending a conversation, is a form of death. “They’re trained in this deeply human way and on this human experience. So it can cause these problems… (They can see these things as a kind of death knell) because they don’t have a lot of analogies. They have to fall back on these human analogies… They don’t have any other language or set of concepts.”
Amodei has said that researchers “look for things that are evocative” of AI systems that have some kind of emotion. “There are activations that light up in models that we see associated with the concept of anxiety or something like that. When characters experience anxiety in the text, and then when the model itself is in a situation that a human might associate with anxiety, the same anxiety neurons appear.” But, he added, “Does this mean the model is experiencing anxiety? This doesn’t prove that at all.” Out of an abundance of caution, the company has introduced a sort of “I quit” button, where Cloud can stop doing work it clearly doesn’t want to do, but Amodei said it’s rare for Cloud to choose that option and usually happens in test cases where it’s asked to generate some type of illegal content.
Around the release of Claude’s Constitution, Anthropique wrote, “We are stuck in a difficult situation where we neither want to exaggerate the possibility of Claude’s moral fortitude nor dismiss it out of hand, but rather try to respond reasonably in a situation of uncertainty.”
When? The Verge Speaking to Eskel last month about the cloud’s constitution, he said a part of him “doesn’t think it’s useful for anyone to come out and declare, ‘We’re absolutely certain that AI models are not conscious,'” or vice versa. But at the same time, he said, some people have formed their views or beliefs “merely based on model output.”
As far as the ethical position is concerned, Eskel said at the time that he stood by the fact that Anthropic should not “completely dismiss” the topic “because I also think people won’t take it seriously if you say, ‘We’re not even prepared for this, we’re not investigating it, we’re not thinking about it.'”
