Confirmation bias: how to catch it in your own decisions

A team I worked with once spent five months building a feature they were certain would fix retention. They were not careless. They had run twenty user interviews. They had a cohort analysis. They had a competitor who had shipped something similar and was growing. Every piece of evidence pointed the same way, and they moved with the calm of people who had done their homework. The feature shipped. It moved nothing. Retention sat exactly where it had been, and the five months were gone.

What went wrong was not the data. The data was fine. The problem was that they had only ever gone looking for the data that agreed with them, and their brains had quietly filed away everything that did not. That is confirmation bias, and it does not feel like bias from the inside. It feels like being right.

This piece is about that one failure specifically. What confirmation bias actually is, why the smartest and most analytical people fall for it hardest, the exact shapes it takes in real work, and a small set of moves that force your own blind spots into view before they cost you a quarter.

What confirmation bias actually is

Confirmation bias is the tendency to seek out, notice, and remember information that fits what you already believe, while ignoring, discounting, or explaining away information that does not. It works on all three ends at once. You go looking in the places likely to agree with you. When mixed evidence arrives, you read the ambiguous parts as support. And weeks later, the confirming facts are the ones you can still recall, while the awkward ones have faded.

The word "bias" makes it sound like a character flaw, and it is not. It is your brain conserving effort. Building a belief is expensive, so once you have one, your mind treats it as settled and fits new information into it rather than tearing it down and starting over. Most of the time that is a fine trade. You do not rederive your worldview every morning. But when you are making a real decision, that same reflex quietly stacks the evidence in favor of whatever you already wanted to do.

Confirmation bias does not feel like ignoring the evidence. It feels like being unusually thorough about the evidence that happens to agree with you.

The classic demonstration is from Peter Wason in the 1960s. He gave people the number sequence 2, 4, 6 and asked them to discover the rule behind it by proposing their own sequences, which he would label as fitting the rule or not. Almost everyone guessed a rule like "even numbers going up by two" and then tested it only by offering sequences that fit that guess: 8, 10, 12. Yes. 20, 22, 24. Yes. They took each yes as proof. The actual rule was just "any three increasing numbers," and the only way to find it was to try a sequence you expected to fail, like 1, 2, 3. Hardly anyone did. That is the whole bias in one experiment. We test our beliefs by looking for wins, when the only test that teaches you anything is the one you expect to lose.

Why smart people fall for it hardest

Here is the uncomfortable part. Intelligence and analytical skill do not protect you from confirmation bias. In some ways they make it worse, because a sharp mind is a better lawyer. Give a clever person a conclusion they are attached to and they will build a more convincing case for it, find more supporting data, and wave off objections more persuasively than anyone else in the room. The skill that should catch the bias gets recruited to defend it instead.

This is why "just be more objective" is useless advice. Nobody thinks they are being subjective. The team that burned five months felt maximally rigorous the entire time. They were not lying to anyone, least of all themselves. They were reasoning carefully from a starting point they never thought to question, and careful reasoning from a bad starting point gets you confidently to the wrong place. The fix is not more effort or more honesty. It is structure that forces you to look where you would rather not.

What it looks like at work

Confirmation bias almost never looks like someone throwing out inconvenient data. It looks like a team being extremely diligent about the wrong slice of it. These are the costumes I see most.

The interview highlight reel

You run twenty user interviews. Three of them mention the exact pain point that matches the feature you already want to build. Your summary leads with those three, quotes them by name, and files the other seventeen under "also noted." You did not fabricate anything. Every word in the writeup is true. But you built a highlight reel of the evidence that agreed with you and called it research.

The metric that happens to fit

You pull the numbers and find one cohort where they support your thesis. That is the chart that goes in the deck. The four cohorts where the pattern does not hold never come up, not out of dishonesty, but because your eye slid past them looking for the one that helped. Worse, you read a correlation as proof of your story when it could just as easily mean the reverse. Motivated users take more actions and retain better on their own, with or without the thing you want to build.

The objection that gets buried politely

Someone raises a real concern in the planning meeting. "What if power users find this annoying?" You say "that is a good point, but," and the "but" quietly buries it. The concern gets addressed procedurally, we will make it dismissible, without anyone ever engaging with whether it means the whole idea is wrong. Everyone leaves feeling the objection was handled. It was handled the way a bouncer handles someone: shown the door.

The convenient competitor

A competitor ships something similar and their CEO posts about record engagement. Into the pitch it goes as evidence the idea works. Nobody checks whether their market, their scale, or their users make the comparison meaningful, because the comparison is convenient and convenient evidence rarely gets frisked. You would never accept that reasoning about a claim you disagreed with. You accept it here because it points the way you already wanted to go.

None of these are acts of bad faith. They are pattern-matching by people under pressure to ship, who genuinely believe in what they are building. The result is the same either way. You build the wrong thing, thoroughly, and with a paper trail that makes it look like a careful decision.

How to actually catch it

You cannot out-discipline confirmation bias, because the bias runs underneath the part of you doing the disciplining. What works is building a few structured moves into how you decide, each one designed to drag disconfirming evidence into the light where you have to look at it. None of these depend on you being more objective in the moment. That is the point.

Ask what would have to be true for the opposite

This is the cheapest and most powerful one, and it is a single question asked before you commit. "What would have to be true for the opposite conclusion to be right?" If you believe social features will lift retention, ask what would have to be true for them to hurt it. The answer comes fast once you make yourself ask: maybe your power users stay precisely because the product is a focused, solo tool, and social noise is the thing that drives them off. You have not changed your mind. You have just forced yourself to spend one honest minute on the case against, which is one more than most decisions get.

Run a pre-mortem

Before you commit, imagine it is six months from now and the bet failed completely. Not "hit some snags," failed. Now work backward and write the story of what went wrong. Have each person write theirs alone before anyone shares, so the room does not converge on one comfortable narrative. The failure stories are where the buried assumptions surface. "We assumed users would find the feature on their own." "We assumed the problem was onboarding when it was actually pricing." A pre-mortem gives people permission to voice the doubt they were sitting on, because now doubting is the assignment.

Assign someone to argue against it

Pick one or two people and give them the job of making the strongest possible case for not doing this. Not contrarian sniping for sport. The real steelman, with data, given time and legitimacy. Make it explicit that poking holes is a contribution, not obstruction, because otherwise the social cost of dissent keeps everyone nodding. A good red team does not kill ideas. It kills the bad ones early and makes the survivors stronger, which is exactly what you want before you spend a quarter.

Check the base rate before you project impact

You have inside information about your bet: your effort, your conviction, your clever plan. What you underweight is the outside view, how often bets like this actually pay off in general. Before you project a thirty percent lift, find the base rate for this kind of feature in your category. If similar features typically deliver a five to ten percent lift that decays by month three, your thirty percent projection now owes everyone a specific explanation for why you will beat the field by three to six times. Usually you do not have one, and the number quietly right-sizes itself.

Keep a decision journal

Before you decide, write down what you are deciding, what you expect to happen, why, how confident you are as an actual number, and what specific result would prove you wrong. Then set a date to reread it. The journal does two things confirmation bias hates. It pins your prediction in writing so you cannot later remember having been right all along, and it names in advance the evidence that would change your mind, so when that evidence shows up you are far less able to explain it away. Over a year of entries you start to see where your judgment is reliably off, which is the raw material for getting better.

A short audit to run before you commit

The next time your team is about to lock in a real bet, run these five questions in writing, alone, before you discuss as a group. Doing it in writing matters, because out loud the group converges on the confident version before the doubts get a chance to form.

1. 1What evidence did we actively seek out that contradicts this bet? If the answer is none, we have not done the work. We have done a highlight reel.
2. 2If this fails completely in six months, what is the most likely reason? Write three failure stories. The one that makes you most uncomfortable is the one worth investigating.
3. 3What is the base rate for this kind of bet in our category? If we do not know, we find out before we project impact, because our inside view is almost certainly rosier than reality.
4. 4Who disagrees, and did their objection get real airtime? Not "we acknowledged it and moved on." Airtime where the dissent actually shaped the plan.
5. 5What specific, measurable result would make us pull the plug, and have we committed to checking on a date? No kill criteria means no accountability. You will rationalize any result you get.

The decision might still come out the same. You run the audit and you still build the feature. But now it is a calibrated bet with a tripwire, not a conviction wearing the costume of analysis. That difference is the whole game. It is also close cousin to the kill criteria that keep you out of the sunk cost fallacy later, when the bet is running and quitting starts to feel like defeat. The same confirmation bias that talked you into the bet will work overtime to keep you funding it.

Why reading this will not be enough

Here is the honest catch. You can finish this article, nod along, fully agree that you should hunt for disconfirming evidence, and then walk into next week's planning meeting and build a highlight reel without noticing you are doing it. I have. Knowing about a bias and catching it live are different skills, the same way knowing about logical fallacies does not stop you from committing one in a heated argument. The knowledge sits in the calm, reflective part of your mind. The bias operates in the fast, automatic part that is already running the meeting before the reflective part shows up.

Closing that gap takes reps, not more reading. You have to practice the moves, spotting the buried objection, asking for the opposite, checking the base rate, enough times that they fire on their own in the moment that counts. Real work is a poor training ground for this, because the bets pay out slowly and tangled up with luck, and you rarely get clean feedback on whether your reasoning was sound or you just got lucky. That missing feedback is exactly what lets the bias survive. It is the same reason overthinking is so hard to shake once it sets in, which I get into in the piece on how to stop overthinking decisions.

Go find the evidence you have been avoiding

You do not beat confirmation bias by resolving to be more open-minded, any more than you get fit by resolving to be stronger. You beat it one uncomfortable look at a time. So take a belief you are holding right now, some bet, some hire, some strategy you feel sure about, and go looking for the strongest evidence that you are wrong. Not to punish yourself. To find out.

Ask what would have to be true for the opposite to be right. Read the four cohorts you skipped. Give the objection you buried its real airtime. It will feel slightly wrong, like you are undermining your own case, and that feeling is the bias defending itself. Do it enough and it flips. You start to feel the pull of the convenient conclusion before it grabs you, you go hunting for the disconfirming fact by reflex, and you make fewer confident, expensive mistakes. If you want a broader map of the skill this sits inside, the guide to improving your decision-making covers the rest. But confirmation bias is the one to start with, because it is the bias that hides all the others.