The AI “Thinking Partner” Lie

Last year I read Johann Hari’s book Magic Pill, about GLP-1 drugs. The book wasn’t for or against, just giving a survey of the landscape and research. I found it fascinating, and a couple of sections about modern food culture struck me as something my kids should read.

Those sections contained vivid descriptions of industrial food processing, and how products like chicken nuggets are formed. He also compared how the Japanese think and teach about food as compared to Americans.

Now, these days my kids are more into Keeper of the Lost Cities and Captain Underpants than narrative non-fiction. But they’re open-minded, and especially if I offer a small cash incentive they’re willing to read a couple dozen pages about something I think they should know.

So I gave the book to the kids, and they sat down to read. It was evening, and once they were done it was time to plan dinner. I asked Priya what she felt like, and she replied firmly and definitively: “Not fast food!”

That was it. She didn’t elaborate. She didn’t need to. The chapters had done their work.

Now, obviously, this wasn’t a permanent fix – she’s wanted fast food plenty of times since then. But that knowledge now lives in her brain in a way that it didn’t before, and it’s a factor in her decision-making – a small counterbalance against the instinctual craving that fast food caters to.

That’s the way the rest of us work too. The technologies and substances that exploit our wiring don’t get neutralized when we learn about them. We still eat fast food. The casinos do fine. Doomscrolling happens at 1am. Knowing about the bad stuff doesn’t stop the bad stuff.

The knowing still has a purpose. It makes us not quite believe ourselves when we’re doing it. The McDonald’s customer knows that the food in the bag isn’t healthy. The casual gambler doesn’t really believe they’re going to beat the house. The late-night scroller knows the algorithm is not their friend. But the gap is real, and it’s how change can find its way in.

But why do we have these vulnerabilities at all? Why is the same brain that figured out fire and language so easy to push around by sugar and slot machines and notifications?

Efficient, Not Lazy

Exercised, by Harvard paleoanthropologist Daniel Lieberman, opens with an interesting puzzle:

If exercise is good for us, why are we so disinclined to do it?

Our evolved instincts and desires are supposed to push us toward what is good for us. And there’s almost nothing that’s as good for us as exercise – and yet it’s one of the hardest things to get ourselves to do! That’s a lot of friction for a behavior that’s supposed to be wired into us.

Lieberman’s answer is that we evolved in a different environment. In our evolutionary past, just being alive took a lot of physical work. We got the exercise we needed as a byproduct of staying alive. We didn’t have to schedule it. And calories were scarce, so we evolved to avoid unnecessary exertion.

But our modern reality is different. These days, you can spend a whole day barely moving and still have a surplus of calories within arms’ reach. But that doesn’t mean we’re lazy. Lieberman’s reframe is we’re efficient. We’re doing what we evolved to do, in a world that no longer exists. The environment has changed, but our wiring hasn’t.

That same pattern shows up in almost every successful exploitation of human wiring – a mismatch between the environment we evolved for and the environment we’re in now. Sweet things (like fruit) used to mean vital calories and vitamins. They were rare windows of opportunity, so we evolved to want them more. The wiring that made foraging work, where the next bush might pay off and might not, is the same machinery slot machines run on. And the wiring that made small-group cooperation possible, where the approval of 150 people could decide whether you ate, is what social media algorithms weaponize against us.

The deeper the gap, the more dangerous the technology that rides it. And one of the deepest gaps we face right now isn’t about what we eat or how we move. It’s about thinking itself.

What Thinking Really Costs

Thinking is metabolically expensive – pound for pound, the brain burns 10x more calories than the rest of your body. So the same logic that made us conserve physical effort applies to mental effort, for the same reason. Thinking harder than you need burns calories your ancestors couldn’t spare.

Susan Fiske and Shelley Taylor named this in 1984. They called it the cognitive miser. The brain looks for the cheaper path and takes it. We accept the available answer over the better one because the available answer is right here. Pattern-matching wins out over reasoning from scratch. Settling for what works wins out over chasing what’s best.

This isn’t a flaw in any individual brain. It’s the architecture that every human brain runs on, including yours and mine.

Robert Cialdini’s body of work on influence is essentially a catalog of levers that exploit this conservation instinct – social proof, authority, reciprocity, scarcity, are all ways the brain reaches for the cheaper path. The cognitive bias literature comes at the same observation from a different angle: confirmation bias, the availability heuristic, sunk cost, all the rest. The brain saves itself work, even when the saving costs us.

For most of human history, the trade-offs of cognitive shortcuts were reasonable. You’d accept your village elder’s view of how to plant the crops. Most of the time it would work. When it didn’t, you’d find out and adjust, and the village would have a slightly better answer the next time around. The shortcuts were built for a world where the cost of bad answers showed up fast enough to fix.

That world isn’t this one. And the technology most capable of exploiting this wiring is also the one that’s hardest to recognize while it’s doing it.

When Failure Looks Like Success

That technology is AI, and it’s strange in a way the others aren’t.

With sugar and slot machines and social media, we eventually figured out what was happening. We produced cultural antibodies to minimize the harm. It took a while, and the antibodies are imperfect. People still engage in unhealthy behaviors. But they know they’re doing it. So there’s a gap between their behavior and the stories they tell about themselves. That gap is the toehold through which reflection or feedback or someone who loves you can reach you.

AI doesn’t have that gap yet, and not just because it hasn’t been around long enough. The real issue is that the failure mode and the success mode look exactly the same.

Consider the line that has become the exhortation of anyone who’s had an AI awakening, to “use it as a thinking partner.” I’ve said that phrase myself. So has just about everyone else who teaches AI. But I’ve come to hate that phrase – because it means everything, and therefore means nothing.

The person who uses AI as a glorified search engine? They used it as a thinking partner.

The person who ran a complex prompt someone gave them, and accepted the first response it gave them? They used it as a thinking partner.

The person who pushes back five times on the model’s first answer and ends up somewhere they couldn’t have reached alone?

All three would describe what they did as ‘using AI as a thinking partner.’ All three would feel like they were. And unlike other modern vices that exploit our paleolithic wiring like fast food or gambling, there’s no scale, sick feeling, or morning when the math comes due. With AI, the failure mode is invisible. “Use AI as a thinking partner” sounds like guidance, but it isn’t doing real diagnostic work – it covers good use and bad use equally well. Everyone hears it and thinks “yes, that’s what I’m doing.”

This isn’t an issue of being careless or reckless with technology, it’s an issue of false positives. The people who are paying attention, who have read the books, who have invested in figuring out how to use this technology well... those are the people I keep watching get caught off guard. They look at their AI use today, compare it to last year, compare it to before AI existed at all, and conclude they’re getting better at this. The trajectory feels right. The output is more than they could have produced alone. And “thinking partner” seems to describe what they’re doing.

It’s like a climber who buys the recommended gear, watches the videos, clips in carefully. The carabiners look right, the procedure feels right. There’s no reason to doubt any of it. Except the carabiners are broken. They have the same shape, with a fresh coat of paint. But the metal underneath won’t hold the load. They won’t notice until the moment they need them to hold, and by then it’ll be too late.

That’s where most people are with AI right now. The phrase we’ve all been teaching – myself included – is the fresh coat of paint.

The Ones Who (Think They) Already Know

I run a lot of AI trainings, and the feedback comes in two flavors.

The first is from people who have barely touched AI before. They show up curious, sometimes a little nervous (which is fair), and they leave saying we blew their minds. That’s what I’d expect.

The second is from people who have been using AI for years. Some of them even teach it. They come into the training confident, sometimes very confident, that they’ve worked out how to use this technology well. They leave saying the same thing, that we blew their minds. Because the version of “using AI well” they came in with turned out to be much shallower than they thought.

The second group was a surprise to me. Because in most fields, if you’ve been at something for years, you’ve developed a reasonable read on what you’re good at and what you’re not. The feedback shows up over time. You learn what you don’t know.

This isn’t happening with AI. Experienced users keep being surprised by what they didn’t know they didn’t know. And it’s not because people using AI happen to be unusually unreflective. It’s a sign that the felt sense of competence isn’t lining up with what’s actually true.

Those are the climbers with the broken carabiners. They’ve done what they think they’re supposed to do. The procedure looks right, the output looks right, and then they discover the gap they hadn’t been able to see.

The early evidence is starting to come in, and it points where you’d expect. Cognitive effort drops. Retention drops. The capacity for independent reasoning erodes – even when the output looks fine. The atrophy is real. It just isn’t visible from inside the chair.

The answer isn’t to stop using AI. It isn’t even to use less of it, or to find a better catchphrase than “thinking partner.” That’s the easier work, and it’s the wrong work. The actual work is to stop trusting how it feels.

Because the way it feels is the same whether you’re using AI well or using it badly. The system can’t tell you. And from inside the conversation, you can’t reliably tell either.

You need something outside yourself to do what your body used to do. Whether that’s a structure, or a practice, or just a way of catching yourself when the gear gives. I’ve written about what that looks like in practice. The Clarity Cascade covers the frame, and my new book AI Curious goes further.

The most dangerous moment is when you read this and feel certain it’s about somebody else.

That feeling is the false positive.

It’s the same feeling you have inside the chair, looking at your last AI conversation and concluding that you used it well.

That might be true. But there’s no way to know from where you’re sitting.

If you’re enjoying these essays, my new book AI Curious goes much deeper into the thinking behind them. It’s not about prompts or tools - it’s about learning to think with AI in a way that makes you sharper, not dependent. Thousands of people have gone through the methodology, and their experiences are woven throughout. Get your copy here →