The Reductionist Position on AI by Cal Newport

Sascha

I send this mail to Cal Newport. I thought that it might be interesting to discuss:

Hi Cal,

tl;dr: You are positioning yourself for failure by taking a reductionist position on AI. (Doesn’t mean that you are wrong. Just your thinking is set up to for failure here)

I enjoyed your last newsletters about AI, as you have a more down-to-earth approach than the typical speculative takes.

However, as AI advances, you run into the same problems that reductionists run into when they try to reduce consciousness to a network of neurons: You are explaining the mechanistic/deterministic substance of an emergent phenomenon (my old professor conclusively rejected the notion of emergent phenomena, as he was a reductionist).

For example:

The most obvious is that once trained, language models are static; they describe a fixed sequence of transformers and feed-forward neural networks.

Brett Weinstein is talking in general terms, not a bout the current reality. The statement by you quoted above will be outdated when AI will be allowed to be more malleable in its substance and self-modify. (One could argue that, since we don’t build new neurons aside from a few brain areas, any content modification of the AI substance is similar enough to the brain that it doesn’t negate consciousness)

Weinstein’s approach, by contrast, is fundamentally pre-modern in the sense that he never attempts to open the box and ask how the model actually works.

If we open our own boxes, we just see glibber. If we look a bit closer, we see just a bunch of chemicals, neurons, axions etc. The brain, its complexity aside, works pretty boringly. It was the same trap of neuroscientists stating that you open the skull and won’t find any consciousness, so it isn’t there.

There is even a note in Luhmann’s Zettelkasten that rhymes on that notion:

Ghost in the box? Spectators visit. They get to see everything, and nothing but that - like in a porn movie. And the disappointment is correspondingly high. https://zettelkasten.de/posts/luhmanns-zettel-translated/#9_8,3

What you are proposing has been done often and failed accordingly. Even if you take a reductionist position, it won’t get you anywhere the same way that no reductionist, not even Daniel Dennett himself, will be practically consistent with his reductionist belief (or rather the rationalisation that leads to such a belief). He will say that he is a reductionist, but will say “I love you.” to his wife and will raise his children to have high agency.

Live long and prosper
Sascha

PS: Please have me on your podcast to set the record straight regarding the Zettelkasten Method.