How to evaluate a UX researcher

The request that contains its own answer

“Run some user interviews and tell us whether people want this feature.” It arrives from someone senior, it sounds reasonable, and as stated it's unanswerable. Interviews can't tell you whether people want something, only what they say about wanting it.

The most valuable thing a researcher does happens in the next thirty seconds. A weak one schedules eight interviews. A strong one asks what decision this is meant to inform, and whether any answer would change it. If the feature is being built either way, the study is theatre with a budget.

Method following the decision, not the budget

Give them a question and a constraint, and watch which one moves. Researchers under pressure often reach for the method they have access to rather than the one the question needs, then narrow the question to fit and don't mention it.

Strong answers name the trade: “with five interviews I can tell you why, not how many; if you need how many, this is the wrong instrument and I'd rather say so than dress it up.” That sentence is worth more than any portfolio.

Spotting a flawed study, including their own

Hand them a study with a flaw in it (a leading question, a sample drawn entirely from power users, a survey the interesting group didn't answer) and ask what they'd do with the result.

The usual failure is seeing the flaw and presenting the number anyway, because the number is what was asked for. Then ask about a time one of their own studies turned out to be badly framed. Anyone who has run enough research has one. Someone who doesn't is either new or not looking.

What they refuse to conclude

This is the whole job compressed into one behaviour. It never appears in a portfolio, because portfolios are written after the conclusion survived.

Ask what the data in front of them doesn't support. A researcher who can separate what it shows from what it doesn't, and say what would settle the rest, protects the company from expensive confident decisions. One who can't is a very well-organised source of false certainty.

Delivering the finding nobody wants

Research that contradicts a decision already made is the most valuable output the function has, and the hardest to deliver. Put them in front of a stakeholder who has committed publicly and ask them to write the message.

There are two ways to fail: bury it in a deck at slide forty, or lead with the contradiction and lose the room. The good version leads with what the person cares about, gives the finding plainly, and proposes a cheap next step rather than a reversal.

What about tooling and methods knowledge?

Check it briefly and separately. Whether someone knows a specific analysis package or has run diary studies can be verified in ten minutes, and it's the least likely reason a research hire fails.

Research hires fail because the work was rigorous and answered the wrong question, or because nobody acted on it. Neither is a methods problem.

Frequently asked

Do I need a researcher, or can a designer do it?

A designer can run sessions. A researcher adds knowing which questions a method can answer and refusing the ones it can't. That's what stops a company spending a quarter on a confident wrong answer. If nobody on your team can say “this study won't settle that”, what you have is interviews.

How do you evaluate a researcher without a research background?

Ask them to explain, in plain language, what a study of theirs couldn't tell you. Rigour is hard to judge from outside; candour about limits isn't. Someone who volunteers the limits of their own work without being pushed is showing you the thing that matters most.

Should I ask for a research portfolio?

Look at one for fifteen minutes to confirm they can structure and communicate a study. Don't use it to rule anyone in. It's written after the outcome is known, with the badly framed studies removed, and those are exactly what you needed to see.