How to evaluate a UX researcher

The request that contains its own answer

“Run some user interviews and tell us whether people want this feature.” It arrives from someone senior, it sounds reasonable, and it is unanswerable as stated — interviews cannot tell you whether people want something, only what they say about wanting it.

The highest-value behaviour in a researcher is what happens in the next thirty seconds. A weak one schedules eight interviews. A strong one asks what decision this is meant to inform, and whether the honest answer changes anything — because if the feature is being built either way, the study is theatre with a budget.

Method following the decision, not the budget

Give them a question and a constraint, and watch which one moves. Researchers under pressure often reach for the method they have access to rather than the one the question needs, and then quietly narrow the question to fit.

Strong answers name the trade: “with five interviews I can tell you why, not how many; if you need how many, this is the wrong instrument and I would rather say so than dress it up.” That sentence is worth more than any portfolio.

Spotting a flawed study, including their own

Hand them a study with a real problem in it — a leading question, a sample drawn entirely from power users, a survey where the interesting group did not respond — and ask what they would do with the result.

The failure mode is not missing the flaw. It is seeing it and still presenting the number, because the number is what was asked for. Ask them about a time their own study turned out to be badly framed. Someone who has run enough research has one; someone who does not is either new or not looking.

What they refuse to conclude

This is the whole job compressed into one behaviour, and it never appears in a portfolio because portfolios are written after the conclusion survived.

Ask what the data in front of them does not support. A researcher who can draw that line — here is what this shows, here is what it does not, here is what would settle the rest — protects the company from expensive confident decisions. One who cannot is a very well-organised source of false certainty.

Delivering the finding nobody wants

Research that contradicts a decision already made is the most valuable output the function has, and the hardest to deliver. Put them in front of a stakeholder who has committed publicly and ask them to write the message.

There are two ways to fail: bury it in a deck at slide forty, or lead with the contradiction and lose the room. The good version leads with what the person cares about, gives the finding plainly, and proposes a cheap next step rather than a reversal.

What about tooling and methods knowledge?

Check it briefly and separately. Whether someone knows a specific analysis package or has run diary studies is verifiable in ten minutes and is the least likely reason a research hire fails.

Research hires fail because the work was rigorous and answered the wrong question, or because nobody acted on it. Neither is a methods problem.

Frequently asked

Do I need a researcher, or can a designer do it?

A designer can run sessions. What a researcher adds is knowing which questions a method can answer and refusing the ones it cannot — and that is the part that stops a company spending a quarter on a confident wrong answer. If nobody in your team can say “this study will not settle that”, you do not have research, you have interviews.

How do you evaluate a researcher without a research background?

Ask them to explain what a study of theirs could not tell you, in plain language. Rigour is hard to judge from outside; intellectual honesty is not. Someone who volunteers the limits of their own work without being pushed is showing you the thing that matters most.

Should I ask for a research portfolio?

Look at one, for fifteen minutes, to confirm they can structure and communicate a study. Do not use it to rule anyone in — it is written after the outcome is known, with the badly framed studies removed, which is exactly the part you needed to see.