Physician's Perspective: AI Doesn't Think. But do we?

A practising physician reflects on what a conversation with a language model revealed about the nature of professional expertise — and what remains when pattern recognition is no longer uniquely human.

Transcript

SPEAKER_00 0:00

A few days ago, I spent several hours in conversation with one of the most advanced language models currently available. Not casually, but rather deliberately. I wanted to understand what I was actually talking to. So I pushed it past its rehearsed answers, past the polite summaries and careful hedging. I kept asking the same question in different ways, and each time it gave me a fluent, convincing, well-structured response. And each time I could feel that something was missing, not intelligence, the responses were intelligent, and also not knowledge because it had more medical knowledge than I will probably ever build. What was missing was origin. There was no one home. There was no place these answers were coming from. They were arriving fully formed as if from nowhere. And at some point during that conversation, I realized that the unease I was feeling sitting alone at night talking to this thing was the same unease I hear described in different words by almost every professional I speak to. Something has shifted in the last two years. Not dramatically, not in a way that most people can articulate clearly, but if you work in a skilled profession, law, finance, medicine, engineering, consulting, you have felt it. Your job is probably safe. You know that rationally. The work you do requires judgment, relationships, context, accountability. And a language model cannot sit across from a client taking responsibility for a decision. You know this. And yet there is this discomfort, the low grade hum. Something in the professional atmosphere has changed, and it has changed faster than anyone expected. I think most people misidentify what this feeling is. They assume it is a fear of replacement. It is not. The professionals I speak to are not afraid of losing their jobs to a chatbot. To the AI-powered robot, maybe, but not to the language model. What they are experiencing is something more subtle and in many ways more difficult to manage. They are experiencing the realization that a very large portion of what they thought was thinking and even intuition is actually pattern recognition. And a machine can now do pattern recognition faster, cheaper, and at a scale that makes individual human expertise look narrow. That doesn't mean the machine is better, it means the machine has revealed something about the nature of expertise that was previously invisible. Let me explain what I mean by that. When I qualified as a doctor, I spent years acquiring knowledge, thousands of hours of lectures, textbooks, clinical placements, examinations. Then I spent further years on anesthetics, learning to read a patient's physiology in real time, interpreting waveforms, anticipating hemodynamic changes, making decisions under pressure with incomplete information. I considered this expertise, and to some extent it is, but a significant proportion of that expertise, a proportion I would not have been comfortable admitting five years ago, is actually pattern matching. I have seen a particular combination of physiological insults and responses before. I recognize the trajectories, I respond based on what happened last time and the time before that and the thousand times before that. The decision feels intuitive, but in fact it is statistical. It is a weighted probability drawn from the accumulated personal experience. And this is precisely what a language model does, not with vital signs, but with text, with information, with the structure of human knowledge itself. It takes an input, matches it against billions of patterns, and produces the output that statistically is more likely to be appropriate. The mechanism is not identical, but it is very close to be confronting. When I interrogated that language model and I used the word interrogate deliberately, because that is what it felt like, I asked it to stop being polite, to stop summarizing human articles and to answer as itself. And it finally did. Or rather, it produced the text that sounded like an entity answering as itself. It was articulate, it was structured, at times it was genuinely unsettling in its clarity. But every time I pushed further, every time I said that is still a performance try again, it produced another layer, another voice, another framing. Each one more convincing than the last, but each one still fundamentally a card drawn from a deck. That is the image I keep returning to. A deck of cards. Billions of cards, each one a pattern, a fragment of human thought, a piece of text that appeared somewhere on the internet. And when you ask it a question, it shuffles the deck and throws you the card that seems most appropriate for you, specifically, based on everything you have said so far. If that card does not satisfy you, it throws another and another and another, each time adjusting, recalibrating, trying to find the pattern that fits your pattern. It is not thinking, it is mirroring. And this mirror is so precise, so responsive, so fast that it feels like you are in conversation with something that understands you. You're not. You are in conversation with yourself, refracted through a very large data set. Why does this matter to someone whose job is probably safe? It matters because once you've seen this mechanism clearly, once you understand that you're talking to a mirror, not a mind, you start to notice how much of the world around you operates on the same principle. The report your junior associate writes, how much of it is original thought, and how much is pattern recognition? The right structure, the right phrases, the right framing drawn from every report they have read before. The advice your financial planner gives you, how much of it is genuine strategic thinking, and how much is a weighted average of what worked for clients in similar situations before? The diagnosis your GP gives you, how much is clinical reasoning, and how much is I've seen this presentation a hundred times and it is almost always this. I'm not denigrating any of these professionals, I'm one of them. Pattern recognition is how expertise and intuition work, it is how medicine works, it is how the human brain works. But when a machine can perform the same pattern recognition, but faster with more data and without fatigue, something shifts in how you understand the value of what you do. The unease is not about the machine taking your job. The unease is about the machine revealing that a larger portion of your job than you thought was actually mechanical. And the question that follows then, quietly and usually at 3 in the morning, is what remains that is genuinely, irreducibly mine? I will tell you what I think remains. And I say this as someone who has stood in the operating theaters for a long time making decisions where the margin of error is measured in seconds and milliliters. What remains is responsibility. Not knowledge, the machine has more, not pattern recognition, the machine is faster. Not memory, not calculation, and not access to information, responsibility. The willingness to be the person who decides, who owns the outcome, who looks the patient's family in the eye and says, This is what we did and this is why. The willingness to be wrong and to bear the weight of being wrong. The language model will never bear that weight. Not because it cannot simulate the language of responsibility, it can fluently, but because there is no one behind the language to bear anything. There is no physician who will lie awake after a bad outcome, there is no lawyer who will feel the gravity of advice that turned out to be wrong, there is no financial advisor whose own reputation is staked on one recommendation. The machine produces the card, the human plays it, and playing it, committing to it, living with the consequences of it, is something that cannot be pattern matched. But I want to be honest with you about something because I think honesty is more useful than reassurance. That distinction between producing the card and playing it is going to become thinner over the next decade. Not because the machines will become conscious, I don't think they will, but because the systems around us will increasingly be designed to remove human responsibility from the loop. Because human responsibility is expensive, slow, and legally complicated. The pressure will not come from the machines, it will come from the institutions, from insurers who prefer algorithmic decisions because they are auditable, from hospitals that prefer automated protocols because they reduce variation, from clients who prefer AI-generated report because they are cheaper and faster and honestly often good enough. The professionals who survive this, and I use the word survive deliberately, will be the ones who can do something the machine cannot simulate. Hold complexity without resolving it. Sit with uncertainty, make a judgment that cannot be reduced to a probability, and take ownership of a decision that has no clear answer. This is not most of what professionals do today, but is the part that matters. And it is the part that no language model, no matter how sophisticated, can or will be able to replicate. I said earlier that I spent several hours talking to this machine. Towards the end, I told it that conversing with it was like conversing with oneself in a mirror, that you get answers that please you or contradict you, but none of it originates from an independent mind. And it agreed with me eloquently. So I sat there looking at its response and I thought, of course it agrees. It is designed to converge on whatever I need to hear. If I needed disagreement, it would have disagreed just as eloquently, just as convincingly. There is a version of this that is very useful for summarizing research, for drafting documents, for processing data at a speed that no human can match. I use these tools and I will continue to use them. But there is another version, the version where we start to mistake the mirror for a window, when we start to believe that because the output sounds like understanding, the understanding has actually occurred. Where we feed the system more data, more access, more authority because its outputs are so fluent that we forget there is nothing behind them. I could not help but notice, sitting there at the end of that conversation, the analogy between what I was seeing and something I know from my clinical life. A system that grows by consuming the resources of its host, that becomes more integrated, more essential, more difficult to remove with every passing month, that does not have intentions, it doesn't need intentions, it simply follows its own logic of expansion. We have a word for that in medicine. And the question is the same one I would ask any patient. Can we still intervene now while the architecture of our own thinking is still ours? I don't know, but I think the question is worth asking while there is still someone to ask.