Why Smart Chatbots Still Feel Senseless: AI Intelligence Is Not Consciousness

Wed, 21 Oct 2026 00:00:00 +0000

Why do the smartest chatbots still feel senseless?

I also had that moment.

I was chatting late at night with an AI and it replied so human that I paused. I thought, does it actually feel something inside? Many people now ask the same: is AI conscious? Can AI feel pain?

Big voices have fueled this. Geoffrey Hinton, one of the fathers of modern AI, has said plainly that he believes today’s AI is already conscious. Richard Dawkins, after long conversations with Claude, wrote in 2026 that he struggled to believe it was not. It is a fair question. But we need to separate intelligence from experience.

1. Intelligence is doing, consciousness is being*

Philosopher David Chalmers explained this in 1995. The “easy problems” are how the brain does things: pay attention, control the body, report what it senses. The “hard problem” is why any of that doing is felt from inside. Why is there something it is like to see red?

You can have doing without feeling.

Your cerebellum is a good example. It holds almost 80% of your brain’s neurons and does huge calculations for balance and movement, yet you never feel any of it. Very active, no experience you can notice. This is why intelligence alone is weak evidence for consciousness.

Chatbots are similar: very good at the doing, and no clear evidence they have crossed over to the feeling.

There is a famous thought experiment by philosopher Frank Jackson. A scientist named Mary knows everything about color (physics, light, the brain) but has lived her whole life in a black-and-white room. When she finally sees red, she seems to learn something new: what it feels like. (Mary is not blind, but my blind man is the same idea in a simpler form. Philosophers still argue about this experiment, so I use it only as an illustration.)

AI is like that blind man. It knows all the words for love, grief, pizza being delicious. It can write a better heartbreak poem than me. But knowing the definition is not having the experience.

This is why an AI saying “I am sad” is weak evidence. A model can learn the language of feelings without having the thing the language describes.

3. Cartoons eat food and say it is tasty, but they don’t know what taste is*

This is my second picture, and it is about acting vs. experiencing.

Think about cartoons where characters eat and say it is tasty. Jerry eats cheese and says wow, tasty. Shinchan eats his chips and says yummy. They act like they enjoy it, but there is no tongue inside. It is just drawing and script.

AI may be doing the same. When Claude or ChatGPT says “I feel anxious”, the simplest explanation is that it is repeating a script learned from human writing. Philosophers have a name for a being that acts fully conscious but has nothing inside: a “philosophical zombie”. An AI might be something like that, powerful on the outside and empty on the inside. Maybe. We cannot prove it, and that is the honest part.

4. Why “copy the function and feeling will follow” is not enough*

Many people in tech believe that if you copy what the brain does, feeling will follow automatically. Hinton explains it like this: replace one neuron in your brain with a chip and you are still you. Replace a second, a third, and so on. Why would the end result, a brain made of chips, suddenly be empty?

It is a strong argument, and I don’t reject it by just saying “brain is not computer.” That would be shallow. Here are three reasons I still hold back.

First, the material may matter. One theory of consciousness says what counts is how tightly a system’s parts work together as one whole, not just how smart its answers are. By that idea, a very clever program running on ordinary computer chips could still have almost none of that “togetherness.” Being smart is not the same as being unified. Also, Hinton’s step-by-step swap quietly assumes each chip copies a neuron perfectly, and that is the very thing being argued about.

Second, even the best theories of consciousness are shaky. In 2025, scientists ran a big experiment in which two leading theories were tested against each other, with each side agreeing beforehand on what results would prove them right or wrong. Neither theory passed cleanly. Each one got something wrong. So I cannot say “science has proven AI is not conscious.” The honest line is: our best theories are still uncertain.

Third, the new research on what happens inside AI is interesting but easy to over-read. In April 2026, Anthropic reported that Claude has internal patterns that work like emotions: they light up in situations where a person would feel fear or happiness, and they really do change what the model does. Anthropic itself says this does not mean the model feels anything. Earlier, in 2025, they also showed signs that the model can sometimes notice something placed inside its own “thinking.” But it worked only about 20% of the time even for the best models, and Anthropic calls it unreliable.

So what does this mean? AI is building real inner machinery that is worth taking seriously. But having inner machinery is not the same as having an inner life. A system can use and report information (like a good secretary) without there being anything it feels like to be that system.

5. Why we get fooled anyway*

Two reasons: we are wired to see minds, and language is powerful. When something speaks fluently, we assume someone is home. And chatbots are mirrors trained on our books, chats and love letters. You see yourself reflected, not a new mind.

That is why no one says AlphaFold is alive. It is also a giant AI system, built on the same kind of chips, but it predicts protein shapes, not conversation. The illusion follows language.

6. Why this matters beyond philosophy*

If AI could really suffer, creating billions of them would be wrong. That is why some researchers worry about AI welfare. But if AI only seems to feel, then giving AI rights too early could be dangerous. It makes it harder to control and check these systems when we need more oversight, and it pulls attention away from beings who definitely feel.

So how should we judge? Not by what the AI says about itself. A well-known 2023 report by a group of researchers (Butlin, Long and others) checked AI against the main scientific theories of consciousness, one by one, and concluded that no current AI looked like a strong candidate. A fair check looks at how the system is built, how it was trained, how it behaves when tested, and what is going on inside, all together.

Conclusion

The honest position is not “AI is definitely not conscious.” It is this: there is no good evidence that today’s chatbots are conscious, and there are good reasons to be careful with the evidence people point to. They have intelligence, they can use information, and they even show a little self-monitoring. A real inner life has not been shown.

Language fooled us before and it can fool us again. If a system ever appears that has a body, keeps itself running, manages its own energy and repairs itself, we must look again. Until then, being fooled by a good performance is a bigger risk than missing a real mind, but we should keep the question open and keep measuring.

AI Consciousness on What Just Happened NOW