Different sources, different questions

Picture a leadership team thinking about moving into a new sector.

Their people say the organisation has the capability. The website barely mentions it. An industry report describes a growing market. A potential client says they’ve never thought of the business for that kind of work.

Someone calls the evidence contradictory.

Someone else suggests more research.

Another spreadsheet begins its promising young life.

But the sources haven’t necessarily disagreed. They may have been answering different questions:

• What can we do?
• What have we made visible?
• Is there activity in this market?
• What does this particular buyer associate with us?

The trouble starts when we treat those answers as interchangeable.

A source needs a job description

We often talk about credibility as if it were a permanent feature of the source, like its font.

A government report is credible. A company website is biased. An interview is anecdotal. A dataset is objective. A client said it, so apparently the matter is now settled.

It’s an efficient filing system, and an incomplete way to think.

A company website is a direct source for what the company says about itself in public. It doesn’t show how well the company performs.

An interview can show one buyer’s reasoning in useful detail. It tells us very little about how common that reasoning is.

A national dataset may be rigorously produced and still describe a population, place or period different from the one we want to serve.

A project record can show that the work happened. It may say much less about who did what, whether the result lasted or whether the same approach would work somewhere else.

Accuracy matters. So do relevance, coverage, timing and the connection between the evidence and the claim.

So the question isn’t only, “Is this a good source?”

It’s also, “Is this the right source for the claim we’re asking it to support?”

Match the evidence to the question

None of this means waiting for perfect evidence before anyone’s allowed to act.

It’s a way of avoiding very confident conclusions built on evidence that was answering a different question altogether.

Disagreement can show you where the work is

Back to the leadership team.

They look at the projects behind their people’s confidence and find relevant delivery experience. The external interviews still suggest hardly anyone outside knows about it.

Both findings can be true.

Being able to do something and being known for it are different things.

The next step might be to turn that experience into evidence a buyer would recognise, then test whether it answers a real buying question.

A louder claim won’t close the gap on its own. Nor will another market report. Sometimes the research is finished enough, and the communication just hasn’t started.

For an individual leader, the equivalent might be a strong track record described almost entirely as a list of responsibilities.

The person says, “The market doesn’t value my experience.”

Perhaps.

But first check what the market has actually been shown. A CV full of accountabilities can be accurate and still say very little about judgement, contribution or effect.

Sometimes the evidence really does conflict. Then look at where it differs:

• Who was included?
• Which period was measured?
• How were key terms defined?
• What incentives shaped the account?
• Were the sources looking at the same thing?
• Did the context change between them?

“Conflicting evidence” isn’t the end of the analysis. It’s often where the interesting part starts.

Five links can still be one claim

A press release becomes a news item.

The news item becomes a newsletter.

The newsletter becomes a LinkedIn post.

By the time it reaches your research folder, the claim has had four changes of clothes and looks like a consensus.

Trace important claims back to where they started.

Repetition tells you how far a claim has travelled. Independent confirmation needs a separate route to the underlying fact.

Ask:

• Who first made the claim?
• How could they know?
• What evidence did they give?
• Did the later source do any checking of its own?
• Are the articles all quoting each other in a neat little circle?

Treat dates just as carefully.

A page updated last week may contain figures from three years ago. “Current website” and “current evidence” aren’t the same thing.

Note the period the evidence actually covers. Some facts go off quickly. A procurement deadline, a policy setting or someone’s availability needs checking far more often than the location of a finished project.

Say what you didn’t find, and where you looked

Research rarely clears up every uncertainty before a decision.

What it can do is make the leftover uncertainty specific enough to deal with.

“No capability exists” is a very different conclusion from:

“We found no public evidence of this capability in the sources reviewed.”

The second tells us what was searched, what was found and what we still don’t know. It also leads to a sensible next question: where else would that evidence be?

This matters because research language tends to get more certain as it travels.

“Not identified in this review” becomes “not available.”

“Limited evidence” becomes “no evidence.”

“Worth investigating” turns into “confirmed opportunity” somewhere between the briefing paper and the enthusiastic slide deck.

Write the conclusion only as strongly as the evidence allows.

That’s not being timid. It’s what makes the writing useful.

Try this on one decision

Take one claim that’s doing a lot of work in a strategy, profile or proposal.

Give it five lines:

1. The decision: What will we decide or do differently if this claim is true?
2. The claim: What, precisely, are we saying?
3. The evidence: Which source can answer that specific question, and how would it know?
4. The boundary: Who, where and when does the evidence cover? What remains unknown?
5. The next check: What finding would change our view, and where could we look for it?

Then choose language that matches the evidence:

• Established: supported strongly enough for the decision at hand.
• Indicated: supported, but with real limits.
• Plausible: consistent with what we found, and still needing a targeted check.
• Not established: the available evidence can’t support the claim.

“Worth investigating” can be an excellent result. It just needs a different next step from “ready to commit.”

A good research file lets someone else follow the reasoning, challenge it and see where their own judgement is still needed.

How many tabs you had open is beside the point.

Reading notes

1. Australian Bureau of Statistics, Data Quality Statement Checklist. Used for its underlying fitness-for-purpose principles, including relevance, coverage, reference period and interpretability.
2. HM Treasury, Magenta Book: Central Government guidance on evaluation (2026), section 3.7. Used for guidance on organising findings around evaluation questions and examining conflicting evidence.