The Coming Surge in False Confidence

We are entering a period in which false confidence will spike across the most accountable roles in business, and AI will be the accelerant. Not because the models are wrong more often, but because they are exceptionally good at telling people what they already suspect, in language that sounds considered.

The pattern is already visible in conversations with risk officers, investment committees, and chief executives who have started using large language models as a thinking partner. The framing is usually the same: I wanted to pressure-test my view. What actually happens is subtler. The user arrives with a hypothesis, phrases the prompt in a way that reflects their priors, and receives back a structured, articulate version of their own position. The output feels like validation from a competent outsider. It is not. It is a mirror with better vocabulary.

This matters more in some industries than others. In financial services, where the cost of a confidently held wrong view can run into billions, the risk is acute. A credit committee using AI to stress-test a thesis on a sector exposure will get a polished memo back. If the prompt was framed by someone who already believes the exposure is manageable, the memo will, on average, support that conclusion with more sophistication than the original analyst could have mustered alone. The committee then signs off feeling rigorous. The same dynamic applies in M&A diligence, in regulatory response drafting, in board papers on strategic pivots. The quality of the prose disguises the narrowness of the inquiry.

The roles most exposed are the ones we tend to assume are most protected: senior, experienced, time-pressured decision-makers. Junior staff are often more sceptical of model output because they have been told to be. Senior leaders, by contrast, have spent careers refining instinct, and an instinct that gets articulately confirmed by a machine feels like corroborated judgement rather than echo. The same applies in medicine, in law, in defence procurement, in central bank policy. Anywhere a single person's confidence translates directly into institutional action, the bias loop is now shorter and smoother than it has ever been.

There is a second-order problem worth naming. As more analysis is produced through AI, the distinguishing signal of a good decision-maker shifts. It used to be the ability to synthesise. Increasingly, it will be the ability to frame the question against oneself. The leaders who will look prescient in 2028 are the ones who, today, are deliberately prompting for the strongest version of the view they do not hold, and treating the comfortable answer as suspect. Most are not doing this. Most are using AI the way they used to use a favoured analyst: to produce a clean version of what they already wanted to say.

Boards and executive committees should treat this as a governance issue, not a technology issue. The question is not which model the firm has licensed. The question is whether the decision protocols around significant calls, capital allocation, regulatory posture, talent moves, force a genuine adversarial input before sign-off, and whether that input is itself shielded from the same prompt-shaping problem. A second AI output is not a second opinion. A dissenting human, briefed independently, still is.

The practical implication for senior leaders is uncomfortable. If you have felt, over the past year, that your judgement has become sharper and faster with AI in the loop, the honest test is to ask when the tool last told you something you did not want to hear. If you cannot recall a specific instance, you are not thinking more clearly. You are thinking more fluently. Those are different things, and the gap between them is where the next round of large, confident mistakes will be made.