Intelligence as a Commodity

And The Philosophy of Deep Learning

My Latest Podcast

My latest podcast was with Natural Language expert Sam Bowman, who is professor of Data Science and Linguistics at NYU, currently spending a sabbatical at Anthropic in San Francisco. Sam is a prominent expert in the space of Natural Language. We talked about AI systems such as ChatGPT, the risks they pose, and the challenge of exercising oversight over machines that become smarter than us.

There is an awe and fear about AI. Has it already gone out of control? A recent letter by prominent researchers has called on a halt of at least six months to pause “Giant AI Experiments” for training models more powerful than GPT-4. The letter is well-motivated. I have warned about the risks of AI algorithms some time ago, but I think the genie is out of the bottle this time. It’s a little late.

It evokes concerns of an old problem in AI, called the control problem that was expressed by the cybernetician Norbert Wiener in 1960, where he said: If we use, to achieve our purposes, a mechanical agency with whose operation we cannot efficiently interfere once we have started it…then we had better be quite sure that the purpose put into the machine is the purpose which we really desire and not merely a colorful imitation of it.

Sounds prophetic, doesn’t it?

The Philosophy of Deep Learning

I attended a workshop at NYU over the weekend of March 25-26 titled The Philosophy of Deep Learning. It was organized by Raphael Milliere from Columbia University, together with my philosopher colleagues Dave Chalmers and Ned Block from the NYU Center for Mind, Brains, and Consciousness. The workshop was most timely considering the recent developments in AI.

What do we mean by the philosophy of deep learning? I define it as “the study of the cognitive capacities, computational properties, and societal impacts of AI Machines that use deep neural networks (DNNs) to learn from data.”

The philosophy of deep learning is related to the Philosophy of science and the Philosophy of Mind, which is the study of thinking, intentionality, feeling, consciousness, and sensory perception of the human mind. It is also related to the empirically-driven fields of psychology, cognitive science, neuroscience, and biology. The workshop brought together a diverse group of perspectives on the philosophical foundations of deep learning.

As a scientist, the workshop raised two broad questions for me. The first is “to what extent can humans understand DNNs?”

There is an implicit belief that “explainable AI” will emerge once we understand how models such as ChatGPT work. There’s a lot of research involving “probing” the neural network to try and understand how it represents its knowledge internally. However, this line of inquiry is hopeful that the machine represents the world at the “symbolic” level, like we think we do. But what if its internals are inherently inscrutable to humans because it represents things statistically in a manner humans can’t understand? In which case, we would be building on something we don’t fully understand.

Which brings up a second question, namely, “Are the current foundations of Large Language Models sufficient for creating human-level AI?” By human-level AI, I mean capabilities that meet or exceed that of most humans. From the philosophy of mind perspective, the question is whether the current architecture of DNNs, broadly termed the Transformer, is capable of supporting things like intentionality, feeling, and consciousness. My takeaway is, not yet.

Intelligence as a Commodity

Philosophical questions and risks notwithstanding, AI is in the early stages of a major paradigm shift. The emergence of pre-trained models of the world such as ChatGPT have demonstrated that they are fit for general-purpose consumption. That genie is out of the bottle. People are already building all kinds of applications using them.

Pre-trained models have transformed AI from an application to a general purpose technology. In the process, intelligence is becoming a commodity. Their owners are beginning to charge according to usage, like electricity.

How did this happen? After all, ChatGPT was trained for one specific prediction task using available language data on the Internet: given a sequence of text, predict what comes next. According to Sam Bowman, this task was serendipitous. It was just at the right level of difficulty, that forced the model to really “understand” context and all kinds of other interesting things about the world that we are still trying to unravel.

What is serendipitous is that learning to perform this prediction task enables the machine to tell you why a joke is funny, summarize or interpret a corpus, compose a poem, and all kinds of other things that it wasn’t explicitly designed to do. That’s what makes pre-trained models a commodity, where the intelligence is configurable to any task requiring it. Like electricity. Ironically, however, no one can predict how ChatGPT will respond to an input, not even its creators. No one fully understands how it works.

I say ironic because in the field of AI, the mantra is that the best way to understand something is to implement it as a computer program. We did, but DNNs were motivated by the structure of the brain, which we barely understand, so perhaps it is unsurprising that we can’t understand DNNs either.

That’s where we stand today. Now, there is alarm. How can we control something we don’t fully understand? I asked this very question seven years ago after the Tay chatbot fiasco. What if these machines start doing things we don’t want them to do?  Can we align them with what we really want? I discussed the alignment problem in depth with two of my previous guests, Brian Christian and Stuart Russell, so check out those episodes.

If intelligence has become a commodity, imagine a world where it trades in the futures markets, like oil and electricity. This would have blown Huxley’s mind.