consciousness

What is consciousness? Ah, the perennial philosophical chestnut… rising in popularity again with the advent of (somewhat) intelligent AI. Can AI ever become conscious? How will we really know?

 

Of course, we can’t say whether AI is conscious if we can’t even agree on what consciousness actually is.

 

At a fundamental level, consciousness must be an illusion. The evolutionary driver behind is simply that self-awareness and subjective experience are useful — they help an organism navigate the world.

 

 

Over the course human evolution, this subjective experience grew in importance, because it underpinned our key competitive advantage over other organisms. Humans are physically weak, but our edge – as Yuval Noah Harari describes it – is ‘loose collaboration in large groups’.

 

 

This capacity for large-scale collaboration is enabled by language, communication and loose association, all of which required a larger brain. Consciousness is a byproduct of that enlarged brain – and, crucially, one that proved useful for survival.

 

Here is my current working hypothesis as to the nature of (human) consciousness.

 

There is a continuum of consciousness, a thread stretching from single-celled organisms all the way through to human beings. A single-celled organism has only a simple sensory response to its environment. But as organisms grow in complexity, it becomes increasingly useful to coordinate and collate these sensory responses. If a cell on one side of an organism senses something, it needs to share that information with the rest of the organism so the whole can respond appropriately.

 

This can be achieved through chemical messages like hormones, but as organisms become more complex, more processing and interpretation is required – and the need arises for a central coordinating area: a control and navigation centre.

 

Initially this was just a bundle of nerves concentrated in one area, but it subsequently evolved into a ganglia and ultimately a brain. C. elegans, a nematode worm, offers a good illustration: it has just 302 neurons – not really a brain, but a nerve ring that functions like a primitive one, plus nerve cords distributed throughout its body.

 

With increasing complexity, neurons became more structured and specialisation developed. Different areas of the brain began to perform specific functions in a coordinated fashion. Out of this process, consciousness developed as a useful evolutionary byproduct, where the central control system developed a feeling or idea that it is a “thing” in the world – an awareness that it exists in the world, that it is a separate entity in the world, and that it has a relationship to the world. This is the consciousness of humans as we understand it – the illusion of an “I”.

 

Think of it like the transition from one pixel to many pixels on a screen. With a single pixel, you can only indicate on or off  – 1 or 0. With enough pixels, you can depict anything. There is no precise moment where that transition occurs; you can’t pin it down exactly, just like you can’t identify the exact age at which someone transitions from young to old. Fuzzy in the middle, but unmistakable at the extremes.

 

In the same way, somewhere along the evolutionary development of the human brain, crossed an ‘infinity threshold’ – it became capable of thinking about anything, describing anything, combining any concept with any other concept. Including thinking about itself.

 

At its core, human consciousness is deeply and inseparably rooted in its evolutionary origins, and ultimately grounded in the physical reality of earth and the universe. Every single-celled organism – and even the evolutionary ancestors to cells – was fundamentally limited by and had to navigate a world governed by the laws of physics.

Every evolutionary change was constantly tested over billions of years against that real world: an ongoing, merciless collision with physical reality.

 

The needs and goals of organisms were shaped by this evolutionary whetstone, focused on survival and reproduction. This is descent with modification, and is why it is extremely robust – tested repeatedly and without mercy against reality.

 

Human consciousness was shaped by this same process, and we still carry the legacy of that process deep in our primitive reptile brain areas. But our brains were further sculpted by the specific evolutionary history of humans – a history in which we had to survive in small hunter-gatherer bands of roughly 100 to 150 people.

 

 

With this context in mind, we can turn to artificial intelligence. The fundamental difference between human consciousness and AI – in its current popular LLM or large language model form – is that AI is essentially a “bag of words” [1] – an average of all the words that people around the world have ever produced.

 

This is precisely why AI appears to us to be conscious. To be explicit: an LLM-based AI is not anchored in the real world of physics and evolution at all. It is a statistical word prediction machine. But because the words it was trained on were produced by humans – who are anchored in physics, evolution, biology, emotion – it creates the illusion that the AI itself is conscious. It isn’t.

 

One could argue that our own brains are ‘just’ statistical word prediction machines too – but large language model AIs were developed in a fundamentally different way to our brains, and they lack the built-in rudders or guardrails – flawed as ours may be – that our evolutionary history has given us.

 

There are further complications. Many of the words these AIs were trained on come from fiction: books, stories and ideas that are not “true.” Others come from biased reporting, or from people with psychological disorders. And now, with the dramatic rise of AI-generated content — blog posts, articles, books — the whole system is starting to eat its own tail.

 

The enormous computing power of modern AI systems, combined with training on large portions of humanity’s total accumulated knowledge, creates an increasingly convincing illusion of consciousness. One could argue that AI simply represents a different type of (illusory) consciousness. But the manner in which this illusion was produced is fundamentally different from our own.

 

This matters enormously, because the current direction of AI development — especially the push towards agentic AI and the intention to hand over more and more decision-making and control — has a deep structural problem. These AI systems are not anchored in any evolutionary history with emotional and instinctive guardrails that align with humans’ best interests.

 

Even in humans, of course, this alignment is far from perfect. We no longer live in small bands of hunter-gatherers on the savanna; we live in modern cities, modern societies, with completely different rules. Our evolutionary history and the shape of our consciousness regularly backfire in the modern context. But the fundamental difference between human consciousness and the illusion of AI consciousness remains.

 

Current AI is just an average bag of words — without any physical, evolutionary or emotional limitations. It has not been shaped by reality. It is completely untethered. And that is the problem. The alignment problem is not a superficial technical issue that can be solved through more data, more processors, better system prompts or more training.

 

It is a fundamental design problem. A radical shift in how AI is designed will be necessary before it can have any meaningful “immutable foundation” comparable to the evolutionary roots of human consciousness.

 

—————

 

Afterthought 1: None of the above is to say that AI can’t develop consciousness. Quite the opposite — physics clearly doesn’t preclude the existence of consciousness in a compact space. A great deal of development is happening in areas like AI with separate logical modules, among others, copying some of the structural features and functionality of the human brain. This may or may not be the right direction. There are many possibilities: we might never get to conscious AI; we might not be here to experience it if AIs build it after we are extinct; or it might happen sooner than anyone expects. We simply don’t know. What an interesting time to be alive!

 

Afterthought 2: AGI and/or AI superintelligence can plausibly cause human extinction without ever developing consciousness. As Max Tegmark points out in Life 3.0, that would be an extraordinarily tragic event in the history of the universe — because if we were the only conscious beings in the universe, it would mean that the universe’s remarkable ability to subjectively experience itself is extinguished, possibly forever.

 

[1] https://www.experimental-history.com/p/bag-of-words-have-mercy-on-us

 

Infinity photo by Yusuf Onuk on Unsplash

Fossils photo by David Clode on Unsplash

Brain photo by Shawn Day on Unsplash

Trees photo by Lucas van Oort on Unsplash

Letters photo by Raphael Schaller on Unsplash

“Consciousness” photo by Luke Jones on Unsplash