AI Isn’t I[ntelligent]

When people say “smart”, I think they mean some messy combination of knowledge and intelligence, which are very different things:¹

• Knowledge: the ability to recall facts, information, and skills acquired through experience or education — when useful.²

• Intelligence: the ability to continuously acquire and apply knowledge to build and maintain a predictive model of the world.³

These definitions frame how I think about artificial intelligence (AI), especially the current generation of generative models.

Knowledge measures a current state. It can’t readily be reduced to a single number, but the concept of being more or less knowledgeable — especially within a given context or domain — makes perfect sense. (Who would push back on the claim that Robert Caro is knowledgeable about Robert Moses?) ChatGPT is incredibly knowledgeable.

Intelligence is a process: it’s a time-domain mechanism for increasing knowledge and using it to understand the world. Large Language Models (LLM) have a static learning phase. During this one-time training, they store knowledge by calculating and saving weights — but for so-called AI, this is a discrete phase that ends when the weights are frozen and the model starts being used for inference (i.e., knowledge retrieval). This freezing of knowledge is the antithesis of intelligence. (How “smart” is a person who doesn’t take in new information or change their mind?) So ChatGPT isn’t intelligent at all; it’s not less intelligent than humans, although intelligence obviously exists on a spectrum, but unintelligent in the sense of a light switch that was never turned on.

AI also fails to recognize what it doesn’t know — a direct consequence of lacking the other aspect of intelligence: integrating knowledge into a higher-order predictive model. Everyone’s been that person in a meeting or conversation that’s way over their head. That’s perfectly normal and healthy if you recognize it. But you’ve probably also seen someone in that same situation who’s unwilling to admit it. They don’t understand what’s happening around them, but they’ve had enough exposure to know that sometimes, when someone says X, someone else says Y — and everyone nods approvingly. So they say Y, without actually understanding what it means.

Sometimes Y is useful and insightful, even if the person saying it doesn’t know why. This is what LLMs do, and they’re very good at it because they know so much. But sometimes what they say is nonsense. This is the annoying habit of confidently making up bullshit⁴ — responding with something, anything, in lieu of insight. This is not intelligent behavior, because pattern matching isn’t intelligence.

Something intelligent, when faced with epistemic uncertainty, would instead clarify the other party’s intent or admit it doesn’t know enough about that subject to opine. (Interestingly, you can ask an LLM why it bullshits, and it will give an accurate answer — but if it was actually intelligent it would integrate this knowledge into its model of the world and stop.)

LLMs mimic intelligence with context windows and reasoning that refine results using domain- and conversation-specific context. They also now use real-time search to expand their knowledge and access information that wasn’t part of their original training data. In real time, these techniques let us probe (and expand) a model’s vast knowledge through interaction. But this is more like search refinement than updating the underlying model and thus fundamentally different from intelligence. I think Dwarkesh says it well:

But the fundamental problem is that LLMs don’t get better over time the way a human would…You’re stuck with the abilities you get out of the box. You can keep messing around with the system prompt. In practice this just doesn’t produce anything even close to the kind of learning and improvement that human employees experience.

The reason humans are so useful is not mainly their raw intelligence. [I’d say knowledge.] It’s their [intelligence: their] ability to build up context, interrogate their own failures, and pick up small improvements and efficiencies as they practice a task.

Humans are useful because they’re intelligent; LLMs are useful because they’re knowledgeable — and that distinction is why we complement each other so well. It’s also why I’m fundamentally not worried about AI destroying the world or taking all our jobs. It’s hard to fear something that can’t learn, because by definition it’s always at least one step behind. It’s a Maginot Line in a world of tanks.

However, just because what we currently call AI isn’t intelligent doesn’t mean real AI is impossible. It would simply require a paradigm shift: something like a multi-modal LLM that continuously updated its weights to encode new knowledge and that used a higher-level framework to stitch its knowledge into a generalized, predictive model of the world.⁵

This would be something fundamentally different and raises interesting questions:

• How much should this model index on new feedback (i.e., update its Bayesian priors)?
• Does it attempt to integrate feedback from the entire planet into a single model or start to balkanize itself through specialization?⁶
• If specialization means I have a personalized version of a generic model, how do I benefit from integrating the inputs from others?
• How do I keep from accidentally teaching it to bullshit?
• What happens if specialization turns everyone’s personal AI into a sycophantic life coach?

Any practical solution to these questions feels messy — and likely to become a microcosm of humanity at large: a legion of independent agents built on a mostly shared body of knowledge, yet differentiated by pockets of peculiarities and tendencies shaped by personal and local experience.

In this hopeful vision of real AI, we would have an abundant and diverse population of intelligent beings, each adapted to its own context. In such a world, hard problems would be solved by teams of AIs and people working together — making them much more like us.