Could AI be sentient with Large Language Models ?
AutogenAI’s General Language Engine-1 summarises the article below as:
A summary of Chalmer’s full talk produced by AutogenAI’s General Language Engine-1 is available. Both the article summary and the talk summary took General Language Engine-1 less than half a second to produce. |
As developments in Artificial Intelligence take increasingly giant strides forward, debate perpetuates around whether Large Language Models (LLM), the neural-network power-houses behind modern AI, could be conscious. You may have followed the news of ex-Google employee Blake Lemoine whose work with the company’s Language Model for Dialogue Applications (LaMDA 2) led him to claim that the technology is sentient.
One of the world’s leading philosophers and cognitive scientists, David Chalmers, recently gave a talk on this very subject. You can read a breakdown of Chalmers’ key points in this article and watch the full talk here.
Large Language Models and sentience – what are we talking about?
Language Models are ‘systems that assign probabilities to sequences of text, thereby predicting and generating text completions.’
Large Language Models are ‘giant artificial neural networks, almost all using transformer architecture with multi-head self-attention.’ Simply put, this means huge computing power drawing on a vast matrix of information to generate text after learning statistical associations between billions of words. For an excellent summary of LLMs see this article by AutogenAI’s machine learning expert, James Huckle.
Chalmers takes ‘sentience’ to be synonymous with ‘consciousness’; in turn, sentience = consciousness = subjective experience or ‘phenomenal consciousness’. Thus, a being has subjective experience, or consciousness, if there is something that it is like to be that thing. This means having:
- Sensory experience, e.g. seeing red or blue has a subjective quality
- Affective experience, e.g. feeling pain or pleasure
- Cognitive experience, e.g. there is a subjective experience to exerting cognitive effort and thinking deeply about something
- Agentive experience, i.e. a feature central to being an agent and deciding to act
Sentience is not the same as intelligence and subjective experience is not equal to sophisticated behaviour. Furthermore, sentience is not equal to goal-directed behaviour, nor does it equate to human-level intelligence (often called ‘artificial general intelligence’). Most theorists hold that fish and newborn babies, for instance, are sentient, without having full human-level intelligence.
The challenge for Proponents and Opponents of LLMs as Sentient
Chalmers’ initial intuition is that LLMs aren’t sentient, but he recognises that the answer is not so obvious and proposes the following challenges:
For PROPONENTS of LLM sentience.
If you think that LLMs are sentient, articulate a feature X such that,
- LLMs have X;
- If a system has X it probably is sentient;
- Give good reasons for 1) and 2).
| Potential features could include: | Chalmers’ response: |
|---|---|
| Self-report
This was Lemoine’s main justification: LaMDA 2 reported that ‘‘the nature of my consciousness is that I am aware of my existence. I desire to learn more about the world, and I feel happy or sad at times’.
|
Leading questions will provide convenient answers .
This evidence is equivocal and weak. |
| Seems-Sentient
For example, upon interaction, Lemoine found LaMDA 2 to seem sentient.
|
Humans tend to attribute sentience where it isn’t present, therefore there is little evidence for this claim. |
| Conversational Ability
LLMs give the appearance of coherent thinking/reasoning with impressive causal/explanatory analysis. |
Current LLMs don’t pass the Turing Test (though we may not be far away!).
Furthermore, appearances may be misleading and are, thus, weak evidence for the claim. |
| Domain-General Abilities
LLMs show signs of domain-general (cognitive) intelligence and can reason about many domains. Two decades ago we’d have taken these abilities as prima facie evidence that the system is sentient. |
Knowledge about the LLM architecture, behaviour, and training removes the mystique around these abilities. Chalmers suggests that the evidence for this claim is inconclusive. |
For OPPONENTS of LLM sentience.
If you think LLMs aren’t sentient, articulate a feature X such that,
- LLMs lack X;
- If a system lacks X it probably isn’t sentient;
- Give good reasons for 1) and 2).
| Potential features could include: | Chalmers’ response: |
|---|---|
| Biology
The assertion that consciousness requires biology. |
This view is contentious and Chalmers has argued against it in other works. |
| Sensory perception
Without it, LLMs cannot sense and therefore aren’t sentient. |
This doesn’t account for affective, cognitive or agentive consciousness. Furthermore, LLM+ with sensory perception are developing fast, e.g. vision-language models. Therefore, this view is contentious and temporary. |
| Embodiment
Lack of a body and ability to act means LLMs aren’t sentient. |
Generated text or speech are a kind of act in themselves. Furthermore, LLMs with robotic and virtual bodies already exist. Therefore this view is weak and temporary. |
| World Model
LLMs are stochastic parrots. They do statistical text processing and minimise prediction error; they don’t have genuine understanding, meaning and world-models. |
This view is weak and temporary, as there is some evidence that they already have world models. |
| Human-level reasoning
LLMs make reasoning mistakes, are inconsistent, and lack humanlike planning. |
This is overly high-bar as it wrongly implies that humans have, without exception or fault, consistent and logical reasoning. |
| Recurring Processing
LLMs are feedforward systems, lacking a memory-like internal state that persists between inputs; they are ‘stateless’. ‘Memory’ is required for consciousness. |
This is a fairly strong point but contentious and temporary as not all consciousness involves memory and there are many quasi-recurrent LLMs. |
| Unified Agency (Strongest view for Chalmers)
LLMs lack consistent beliefs and desires, stable goals of their own, and thus aren’t really unified agents. |
While it can be argued that some people are disunified e.g. dissociative identity disorders, they are still more unified than LLMs. Chalmers suggests this is a strong argument against LLM sentience, but may also prove to be temporary. |
What’s the verdict, Could AI be sentient?
Chalmers thinks we can’t decisively confirm or deny the sentience of current LLMs, and that “finding a conclusion counterintuitive or repugnant is not sufficient reason to reject the conclusion”. Thus, we should at least take the hypothesis seriously and the prospect of AI sentience even more seriously.
As recognised by Chalmers, we lack a clear definition and understanding of sentience/consciousness and of LLMs. I believe that until the ontology of sentience and of LLMs becomes a basic belief, i.e. a belief that is justified, but not by its relation to any other beliefs – making it self-evident or self-justifiable – we cannot dismiss either view as irrational. Recognising that both beliefs have warrant, and that they can (and must) exist simultaneously is how and why we can continue to have meaningful discussions about the topic.
AutogenAI’s General Language Engine-1 summarises the article below as:
A summary of Chalmer’s full talk produced by AutogenAI’s General Language Engine-1 is available. Both the article summary and the talk summary took General Language Engine-1 less than half a second to produce. |
As developments in Artificial Intelligence take increasingly giant strides forward, debate perpetuates around whether Large Language Models (LLM), the neural-network power-houses behind modern AI, could be conscious. You may have followed the news of ex-Google employee Blake Lemoine whose work with the company’s Language Model for Dialogue Applications (LaMDA 2) led him to claim that the technology is sentient.
One of the world’s leading philosophers and cognitive scientists, David Chalmers, recently gave a talk on this very subject. You can read a breakdown of Chalmers’ key points in this article and watch the full talk here.
Large Language Models and sentience – what are we talking about?
Language Models are ‘systems that assign probabilities to sequences of text, thereby predicting and generating text completions.’
Large Language Models are ‘giant artificial neural networks, almost all using transformer architecture with multi-head self-attention.’ Simply put, this means huge computing power drawing on a vast matrix of information to generate text after learning statistical associations between billions of words. For an excellent summary of LLMs see this article by AutogenAI’s machine learning expert, James Huckle.
Chalmers takes ‘sentience’ to be synonymous with ‘consciousness’; in turn, sentience = consciousness = subjective experience or ‘phenomenal consciousness’. Thus, a being has subjective experience, or consciousness, if there is something that it is like to be that thing. This means having:
- Sensory experience, e.g. seeing red or blue has a subjective quality
- Affective experience, e.g. feeling pain or pleasure
- Cognitive experience, e.g. there is a subjective experience to exerting cognitive effort and thinking deeply about something
- Agentive experience, i.e. a feature central to being an agent and deciding to act
Sentience is not the same as intelligence and subjective experience is not equal to sophisticated behaviour. Furthermore, sentience is not equal to goal-directed behaviour, nor does it equate to human-level intelligence (often called ‘artificial general intelligence’). Most theorists hold that fish and newborn babies, for instance, are sentient, without having full human-level intelligence.
The challenge for Proponents and Opponents of LLMs as Sentient
Chalmers’ initial intuition is that LLMs aren’t sentient, but he recognises that the answer is not so obvious and proposes the following challenges:
For PROPONENTS of LLM sentience.
If you think that LLMs are sentient, articulate a feature X such that,
- LLMs have X;
- If a system has X it probably is sentient;
- Give good reasons for 1) and 2).
| Potential features could include: | Chalmers’ response: |
|---|---|
| Self-report
This was Lemoine’s main justification: LaMDA 2 reported that ‘‘the nature of my consciousness is that I am aware of my existence. I desire to learn more about the world, and I feel happy or sad at times’.
|
Leading questions will provide convenient answers .
This evidence is equivocal and weak. |
| Seems-Sentient
For example, upon interaction, Lemoine found LaMDA 2 to seem sentient.
|
Humans tend to attribute sentience where it isn’t present, therefore there is little evidence for this claim. |
| Conversational Ability
LLMs give the appearance of coherent thinking/reasoning with impressive causal/explanatory analysis. |
Current LLMs don’t pass the Turing Test (though we may not be far away!).
Furthermore, appearances may be misleading and are, thus, weak evidence for the claim. |
| Domain-General Abilities
LLMs show signs of domain-general (cognitive) intelligence and can reason about many domains. Two decades ago we’d have taken these abilities as prima facie evidence that the system is sentient. |
Knowledge about the LLM architecture, behaviour, and training removes the mystique around these abilities. Chalmers suggests that the evidence for this claim is inconclusive. |
For OPPONENTS of LLM sentience.
If you think LLMs aren’t sentient, articulate a feature X such that,
- LLMs lack X;
- If a system lacks X it probably isn’t sentient;
- Give good reasons for 1) and 2).
| Potential features could include: | Chalmers’ response: |
|---|---|
| Biology
The assertion that consciousness requires biology. |
This view is contentious and Chalmers has argued against it in other works. |
| Sensory perception
Without it, LLMs cannot sense and therefore aren’t sentient. |
This doesn’t account for affective, cognitive or agentive consciousness. Furthermore, LLM+ with sensory perception are developing fast, e.g. vision-language models. Therefore, this view is contentious and temporary. |
| Embodiment
Lack of a body and ability to act means LLMs aren’t sentient. |
Generated text or speech are a kind of act in themselves. Furthermore, LLMs with robotic and virtual bodies already exist. Therefore this view is weak and temporary. |
| World Model
LLMs are stochastic parrots. They do statistical text processing and minimise prediction error; they don’t have genuine understanding, meaning and world-models. |
This view is weak and temporary, as there is some evidence that they already have world models. |
| Human-level reasoning
LLMs make reasoning mistakes, are inconsistent, and lack humanlike planning. |
This is overly high-bar as it wrongly implies that humans have, without exception or fault, consistent and logical reasoning. |
| Recurring Processing
LLMs are feedforward systems, lacking a memory-like internal state that persists between inputs; they are ‘stateless’. ‘Memory’ is required for consciousness. |
This is a fairly strong point but contentious and temporary as not all consciousness involves memory and there are many quasi-recurrent LLMs. |
| Unified Agency (Strongest view for Chalmers)
LLMs lack consistent beliefs and desires, stable goals of their own, and thus aren’t really unified agents. |
While it can be argued that some people are disunified e.g. dissociative identity disorders, they are still more unified than LLMs. Chalmers suggests this is a strong argument against LLM sentience, but may also prove to be temporary. |
What’s the verdict, Could AI be sentient?
Chalmers thinks we can’t decisively confirm or deny the sentience of current LLMs, and that “finding a conclusion counterintuitive or repugnant is not sufficient reason to reject the conclusion”. Thus, we should at least take the hypothesis seriously and the prospect of AI sentience even more seriously.
As recognised by Chalmers, we lack a clear definition and understanding of sentience/consciousness and of LLMs. I believe that until the ontology of sentience and of LLMs becomes a basic belief, i.e. a belief that is justified, but not by its relation to any other beliefs – making it self-evident or self-justifiable – we cannot dismiss either view as irrational. Recognising that both beliefs have warrant, and that they can (and must) exist simultaneously is how and why we can continue to have meaningful discussions about the topic.
