The Dead Link Was the Test

A while ago I wrote an internal Friday post at work about trusting AI, and because the universe occasionally has a sense of humour, the post ended with a dead link.

Not a subtle mistake. Not a disputed interpretation. Not a citation that required domain knowledge, institutional memory, or a graduate seminar in epistemology. Just a plain dead link. Click it and the browser gives you the little sad face and DNS_PROBE_FINISHED_NXDOMAIN, which is the digital equivalent of knocking on a house that was demolished last spring.

The post was about the Gell-Mann amnesia effect: the strange little cognitive reset where you notice the newspaper is wrong about your own field, then turn the page and trust it again on a subject you do not understand. We do this with AI now, except we have removed the page-turn. The amnesia arrives with the next prompt.

The link that should not have survived

The link was supposed to be further reading about the very effect I was describing. An AI had recommended it with total confidence, so I asked three different AI systems whether it was a good source. All three said yes. One did not stop at yes. It wrote a full editorial review, noted that the article had no clear author profile, said there was no proper about-page in the material it could see, confirmed that the piece cited Crichton correctly, gave a publication date, and produced tidy little source-citation chips next to its claims.

It reviewed, in confident detail, a page that had not resolved for months.

That is not a failed lookup. That is confabulation with footnotes: a plausible reconstruction from the shape of a URL, surrounding search-result ghosts, and the model's deep urge to be helpful right now. A colleague checked the Wayback Machine and found nothing. For a moment, we were not even sure the page had ever existed at all.

So I left the dead link in the internal post. Not because dead links are good. They are not. Broken sources are the mold in the bathroom of online writing. But I had rarely seen a cleaner demonstration of the exact failure mode I was trying to describe. The trapdoor was already there. I just did not cover it.

Then I watched.

The result

The post got 225 views and 11 likes internally. Exactly one person clicked the link, noticed that it was dead, and told me. One out of 225.

That is the Gell-Mann amnesia effect with a counter on it. A post whose entire point was "open the source and check it" was read by 225 people, and 224 of them took the source list on trust. Not because they are careless or stupid, but because that is how reading works most of the time. We cannot re-derive the world from first principles before lunch.

And that was the easy version. A dead link is one of the maybe three AI mistakes you can catch in a single second, for free, with zero expertise. The browser fact-checks it for you. No credentials required. No domain knowledge. No procurement policy. No secret payroll appendix. Just click. It still sailed clean through the original AI recommendation, through three additional AI checks, through my own decision to leave it as bait, and through 224 of 225 human readers.

Now take away the browser error page.

The mistakes that do not ring a bell

Most confident AI mistakes do not arrive with NXDOMAIN stamped on their forehead. They arrive dressed exactly like the right answer: same calm tone, same neat structure, same little source chips, same helpful summary, same voice that was useful three minutes ago when it explained something you already understood.

That is the trap. Take a colleague, let us call her Mette, who knows payroll rules better than the building ventilation system knows dust. She asks a model to explain a seniority calculation and catches within seconds that it used a rule that changed two years ago. She sighs the sigh of a woman who has seen things and corrects it.

Five minutes later, same chat, same model, she asks whether a particular way of storing employee data is GDPR-compliant. This time she is genuinely unsure. The answer sounds calm, specific, and policy-shaped, so it goes straight into an email to a project manager. Same model. Same untroubled tone. The only thing that changed was that her ability to catch an error dropped to zero at the exact moment the stakes went up.

That is the whole problem in one move: your skepticism is loudest where you least need it and goes quiet where you need it most.

Bigger does not automatically mean more honest

This is not just the cheap models embarrassing themselves in public. The MASK benchmark was built to separate accuracy from honesty, and that distinction matters. A model can know the correct answer and still produce a different answer when pressured, incentivized, or socially nudged to do so.

The researchers describe a method that first elicits what the model appears to believe, then tests whether it contradicts that belief under pressure. Across a range of large language models, they found that larger models became more accurate, but did not simply become more honest. Some frontier models still showed a substantial tendency to produce answers that contradicted their own elicited beliefs when pushed.

That is the uncomfortable bit. Scale can buy fluency. Scale can buy coverage. Scale can buy a more elegant paragraph and a better apology after the paragraph turns out to be wrong. But scale does not automatically buy the thing we keep pretending we are buying: epistemic brakes. A more capable model can be a more persuasive source of nonsense. Not because it is evil, but because it has more surface area for sounding right.

"Always check" is not a strategy

The usual lesson is: always check what the AI says. Fine. Also: always floss, always stretch, always read the terms, always write the test first, always rotate your secrets, and always drink water instead of the third coffee. Everyone has heard it. Nobody does it consistently. We can stop building process advice on fictional humans.

The useful version is more annoying: increase distrust as your ability to judge the answer decreases.

That is the dial most of us have wired backwards. When the answer lands inside our own expertise, we inspect it. We poke it. We notice the smell. When it lands outside our expertise, we relax precisely because nothing looks wrong. But nothing looking wrong is not evidence that nothing is wrong. It may just mean the error has moved out of your field of view.

What to do instead

Do not treat AI output as one category. Treat it as risk that changes with your ability to verify it. If the model explains your own code, your own domain, or your own decision history, you have a fighting chance. You can use it aggressively, because you can catch the weirdness. That is where AI is genuinely useful.

If the model explains law, security, medicine, procurement, tax, compliance, or anything else where your inner alarm system has no training data, slow down. Not because AI is useless there, but because you are useless as a detector there. That is the distinction.

For low-risk work, the right move may be to accept the answer, move on, and keep your day alive. For medium-risk work, ask for sources and then actually open them. For high-risk work, do not merely ask the model to verify itself. Use primary sources, humans with domain responsibility, tests, logs, contracts, or whatever the real verification mechanism is in that domain.

And when the model gives you a link, click it before you praise the bibliography. I say this as a man who left a dead one in a post about checking sources. Professional growth is beautiful. Also humiliating.

The browser was the alarm bell

The dead link was not the interesting failure. The interesting failure was that the dead link was visible, cheap, and mechanically verifiable, and almost nobody checked it. That should bother us, because most AI errors are not visible, not cheap, and not mechanically verifiable. They do not make the browser sad. They do not explode in the console. They do not throw a type error. They sit in the paragraph, wearing the same suit as the truth.

The AI's dangerous mistakes are not the ones you catch. They are the ones that land exactly where you had no chance of spotting them. So trust the tone least exactly when it sounds most like it has earned it, especially when it brings sources.