Intelligence Isn't the Bottleneck

Neither AI nor regular-I. Plus 100 attention tokens, please.

[To listen to my voiceover instead of reading this post, see the play button above.]

I. Back to the bubble (BTTB)

If you work in a corporate environment, there is a 99% chance you’ve heard some leader or manager say, “We need to figure out how we can better utilize AI for this process.”

Even if you aren’t a corporate bro/gal, you’ve likely heard someone in your social circle say, “Ugh I really need to better use AI so I can save time / accomplish more / upskill in my career.”

If your reflexive response is the following, you are not alone.

Yet, there are reasonable questions you could ask instead that might get you in trouble at work:

Any bull market (and technically the market itself) is propped up by collective group psychology. By extension, a financial bubble is group psychology that has entered into the realm of hysteria (and often delusion). Even though the breakthroughs in generative AI (as seen in products like ChatGPT) are very legit and improving monthly, it’s worth being honest and objective about how useful AI actually is for a given task.

For software engineers, spreadsheet builders, writers looking for typos, or me every time I get inspired to cook after watching this scene from Eddie’s Million Dollar Cook-Off,¹ the benefits of AI are increasingly self-evident.

But for any given knowledge worker being fed top-down dogma, or any random person seeing startup valuations, it’s reasonable to feel like Jacobim Mugatu.

The consummate dogmatic hype man is Anthropic’s CEO, Dario Amodei. He’s written about extreme utopian scenarios and nightmarish dystopian ones involving unchecked AI. The marketing strategy for Anthropic’s new products (i.e. their large language models) is typically the following:

I did put a few of Trump’s words in Dario’s mouth, yes.

It’s effective because Dario means it, and when enough other people believe him, he sees his net worth reach 11 figures. To be clear, Anthropic is a market leader of a very useful product. And to be fair, I’d also hard-sell my company’s vision for a billion dollars. But for the bull market to continue indefinitely, people have to keep accepting these Revelations without asking too many questions.

Thus, if enough people were to ask “Hey is it possible there’s a middle ground between dystopia and utopia if we use your software product?”, and the hype train slowed down, much of America’s recent economic boom might vanish.

II. What’s our problem though for real for real?

Let’s take a few dozen steps back and ask an easier question than “in which month will the computers take over?”

What problems would more intelligence (whether AI or human intelligence) actually solve?

Whether they are our individual problems—a cancer diagnosis, divorce, death of a loved one, medical debt—or humanity’s collective problems—waging war, curing cancer and Alzheimer’s, mitigating extreme climate events, fixing the social safety net—all problems can be essentially divided into two groups.²

III. The counterargument and physical constraints

Since I’m arguing that AI is already good and going to get better, it’s just not going to solve most of our worldly problems, the opposing argument would either be “actually AI entirely sucks and won’t improve” (which I’ll dismiss because that’s dumb) or “actually AI is going to solve pretty much all of our problems because intelligence is the main bottleneck.”

If I were to steel-man that utopian argument, I’d probably just link to Scott Alexander et al.’s viral 2025 essay called “AI 2027.” The basic idea is that AI will (soon) reach a point of intelligence where it can recursively improve itself and then achieve a rapid takeoff towards super-intelligence.

A super-intelligent entity would end warfare as we know it. It would end labor and ensure abundance for all humans. It could reverse climate change. It would somehow convince high-profile men that foregoing $100 million to have an affair isn’t worth it despite their Athleisure Ape’s brain saying otherwise.

Even as a techno-optimist, the reality is that there is limited current evidence that AI is imminently going to solve the problems in the right column above, much less the left column.³

I do believe, entirely subjectively, that AGI (artificial general intelligence) has already been here for some time. If ChatGPT’s breadth and depth of knowledge obviously exceeds any human’s, how is that not “general” intelligence? I also believe that the AGI goalposts keep moving to keep the hype cycle afloat.

For the “intelligence is the biggest bottleneck and AI will solve big problems faster than Paxton realizes” camp, the bottleneck to that rapid AI improvement is, ironically, rooted in the physical world. Between chip shortages, capital constraints, energy inputs, and companies like ASML that have monopolies on specific parts of the supply chain, AI progress can only move as fast as the physical world and human beings allow it to.

IV. The actual bottleneck

I think we’ll find we’re in the singularity and it’ll be like, “Okay, we’ve still got a long way to go.” — Elon Musk (here)

For the majority of our earthly problems—say, providing clean drinking water for all—the bottleneck is not knowledge. We know how to clean water, we just lack the willpower to make sure everyone has it. It’s obviously not as easy as waving a wand, and it’s more a sin of collective omission than collective malice, but it’s also a problem that Anthropic and OpenAI and Google won’t solve with their 2028 releases.

Collective willpower often requires capital and decisions and action. Individual willpower sometimes just requires decisions and action. All of these—willpower, action, decisions, capital allocation—are downstream of the same prerequisite: attention.

Attention, more than intelligence or knowledge or willpower, is the ultimate bottleneck.

When something is urgent—the Manhattan Project, COVID response, etc.—it gets sufficient attention for people to act relentlessly to find a solution. Less urgent but nagging matters in our own lives—say, cleaning out the house of a hoarder relative—only get addressed when we allocate attention.

I use “allocate” intentionally, because no matter how much money you make or how rich you marry or how much Mr. Beast gives you for living in a grocery store for 98 days, we all have a finite number of “attention credits” (we’ll say 100 for easy math) to use each day. You can never roll them over, and they all come from the same bucket.

While the volume of inbound stimulation has increased exponentially this century, our attention credits remain fixed. I think this is partially why—in addition to economic realities—so many people feel “always tired” or are “just so busy.” Instead of having a few buffer attention credits to think, or to rest their brain through boredom, it’s easy to spend 50 credits on work, 20 on short-form video, 10 on scavenging for a meal, 15 with family, two at the gym, and one on listening to this voiceover during your commute.

That’s 98 credits, and it’s impossible to feel activated or mobilize the energy to solve our own problems when we have only two credits left.⁴

Extreme attention fragmentation and exhaustion from distraction really do keep us from solving problems, more often from even getting started. As T.S. Eliot wrote, “We are distracted from distraction by distraction.”

If we summed everyone’s attention-credit lives and asked ourselves what it would take to solve the biggest collective problems facing humans, it would be clear that attention, not knowledge or intelligence, is the ultimate bottleneck.

1. “Winning never tasted so good” is such an incredible subtitle. Disney has fallen off.

2. I choose “knowledge” because one could argue that eight billion more intelligent people would not fight wars or murder or forget to move their bodies. But since new knowledge won’t meaningfully impact something like why we start wars (answer: greed, ignorance, our primitive ape minds, etc.), then AI advances won’t save us. Human nature is still human nature.

I recognize I’m excluding some nuance for the sake of keeping this post short. Knowledge advances like nuclear weapons do deter wars, for instance, but no new version of ChatGPT or Claude will end human psychopathy.

3. Frankly, much of the utopian vision about AI saving everyone does feel like people seeking to fill a religious void.

4. My paranoia about social media is how easy it is to give half of our finite credits away to fuel someone else’s business model. In doing so, we have no credits left to ask ourselves if we actually want to spend our credits that way continually. (To be clear, I don’t think people need to be puritanical. “Everything is fine in moderation, including moderation,” to quote Michael Pollan.)