Why the dawn of AI feels so unsettling

Four reasons people distrust AI, and why they all make sense

The increasingly polarised divide between AI advocacy and scepticism is interesting for a few reasons. While people are often dismissive or wary of new technologies, at least at first, the depth of feeling about AI is on a different order of magnitude.

Here’s what I think could be behind this.

1. An entire generation has yet to experience genuine technology disruption

While it’s true that we’ve seen more technological progress in the last twenty-five years than ever, it’s also fair to say that the last two decades have been relatively orderly and pedestrian compared with the often spiking, lurching tech advancements we saw in prior decades.

For example, take the smartphone, the pre-eminent technological artefact of our time, which arrived with huge fanfare and then promptly reverted to a pattern of incremental improvement, with a steady drumbeat of consumer-friendly incrementalism that enables market analysts to accurately predict the next couple of years’ worth of new smartphone features well in advance. And it’s been much the same for the rest of our modern tech inventory. Broadband connectivity. Camera resolution. Cloud apps. Social media platforms. We’ve become accustomed to each segment delivering a small handful of new features or marginal improvements every year or so.

If the underlying architecture of today’s tech world was largely defined by the mid-2000s, what followed has been more about evolution than revolution. Therefore, anyone who came of age with technology after roughly 2000 has never experienced a paradigm shift; they’ve only experienced acceleration within a stable one.

In this context, the threat model for AI is not “Uber for [insert category]” or “Desktop software but in a browser,” but something much harder to comprehend: intelligence itself becoming a commodity. And, frankly, that does not sit easily with an evaluative framework that two decades of gradual, tamed progress have etched onto our brains. It’s therefore not surprising that this is disorienting and discomforting (and contempt-inducing) for many.

Interestingly, Gen Z (those born between 1997 and 2012), at least according to recent reporting, are both the biggest users of Gen-AI and its greatest sceptics.

2. The product logic is inverted

For more than forty years, the best product thinking has held to a single organising principle: start with the customer problem and work back to the technology. Jobs, Bezos, and every serious practitioner in the tech world faithfully espoused this law: understand the customer’s need first, then build the technology to serve it.

But the Generative-AI era has flipped this script at scale. The capability arrives first — vast, general and only partially understood — and the search for problems comes after. Entire product categories are being constructed not from observed human need but from the desire to deploy a capability that’s demo-ready. When products feel like solutions hunting for problems, people sense it.

3. The industry had already spent its trust

AI did not arrive on a blank slate. It arrived after two decades of surveillance capitalism, algorithmic addiction, platform monopoly, and recommendation systems optimised for engagement and marketing at the cost of everything else. The resulting grievances are legitimate, and so when an industry with that track record announces that its next product is the most transformative technology in a generation, some degree of hostility is understandable and frankly, warranted.

4. “Software brain” has made it personal

Underneath the distrust sits something more specific. Writing in The Verge last week, Nilay Patel beautifully coined a term for it: “Software brain”. In his framing, it describes a particular habit of mind that fits everything into algorithms, databases, and automated loops — one in which, as Patel puts it, Zillow is a database of houses, Uber is a database of cars and riders, and every human preference is a data point to be captured and acted upon.

Patel’s argument is that software brain has always run the tech industry, but AI has turbocharged it, giving more businesses, in more sectors, the ability to automate more of their operations than at any point in history. The result is a fundamental disconnect between how people with a software brain see the world and how regular people live their lives.

As a long-term software brain adherent, to me, that diagnosis rings uneasily true, but it understates what makes this particular moment so challenging for some. AI does not just reflect software brain, it deploys it directly onto lived experience at scale, and without consent. The chatbot that will not transfer you to a human. The hiring screen that filters your CV before anyone reads it, or the recommendation engine that always feels slightly off. These are not neutral tools. They are software brain applied to everyday life, and the cumulative effect is a loss of agency, the feeling that the system is optimising for something, and that something is probably not you.

Thus far, the tech industry’s response has been wide of the mark. Sam Altman thinks it’s a marketing problem, and so last month bought a daily YouTube AI news channel. Satya Nadella says the industry needs to earn social permission. Both treat the hostility as a perception gap that better messaging could close.

But, as Patel says, you cannot market people out of their own experience. The animosity will not recede until a new perspective forms, and people’s evaluative instincts start working again — and until the industry reckons honestly with what software, at scale, actually does to people.

In the meantime, the hostility is worth reading as a signal. Not that AI is overhyped, although that is most certainly the case in some quarters, but that something with the texture of real, disruptive innovation has arrived.

And after twenty years of relative comfort, this new sense of discomfort may just be what real disruption feels like.