How to tell if a design portfolio was generated with AI

Stop trying to detect the text

AI-detection tools have a false-positive problem serious enough that using them to reject a candidate is indefensible. Non-native English speakers and anyone who writes carefully get flagged constantly.

The more interesting case is a project that happened, written up by a model afterwards. The work is the candidate's; the reasoning in the case study was generated. No detector can tell that apart from good editing, and it's far more common than a fully fabricated portfolio.

Six things a generated case study almost never has

Language models write the story a project should have had. Lived projects have friction, and friction leaves traces.

• A constraint that was never resolved. Most projects ship with known problems. Generated ones resolve everything.
• A direction that was abandoned, and why. Models produce a single confident path.
• A disagreement with a named function: engineering said no, legal blocked it, sales overrode it.
• Numbers that are inconveniently precise. 'Activation went from 14% to 23%' reads differently from 'we significantly improved activation'.
• Something the designer got wrong and found out later.
• A decision made for a boring reason: no budget, no data, someone left.

The one question that settles it

Ask for a decision they now think was wrong, and how they found out.

It works because the artefact can't answer it. Getting something wrong means having held a belief, acted on it, and received contradicting information over time. A model summarising a finished project has access to none of that.

Someone who was there answers with a mechanism: support tickets that went the wrong way, a metric that moved and then came back, a user who said something that didn't fit. Someone who wasn't answers in the abstract, with something like 'I would have done more research upfront'.

Why this is getting harder, not easier

Two things are compounding. Portfolios get better because generation improves, and they get more uniform because everyone uses similar tools on similar templates. The variance that used to carry signal is being flattened out.

Better detection won't fix that. The structural fix is to stop relying on artefacts produced privately, in unlimited time, about work you can't verify, and to observe reasoning you prompt yourself, under conditions you control.

The premise is wrong, and it's costing you candidates

Everything above assumes the goal is detection. Stop and ask whether it should be, because the detection framing produces two errors and only one of them is visible.

The visible error is letting through someone who can't do the work. The invisible one is rejecting someone who can, because their writing was clean or their process diagrams were tidy.

That second error happens constantly now and leaves no trace. It falls hardest on non-native English speakers and on anyone who used a tool to fix their grammar, which is close to everyone under thirty.

No detector, human or automated, separates these reliably. Every published test of AI-text detection shows both false positives and false negatives at rates that would be unacceptable in any other part of a hiring process.

So the better question is “does this person have the judgment the work claims?” rather than “was this generated?”. That question has always been answerable. Nobody needed to ask it before, because producing a plausible artefact used to require having done the work.

Assume everything was generated, and design for it

The cheapest reframe available: treat every submitted artefact as if a model produced it, and build a process that still works under that assumption. It costs you nothing, because a portfolio made by a person survives this treatment easily.

What survives is anything that requires having been in the room. The constraint that was never written down. The reason the obvious approach was rejected. What the stakeholder actually said. What got cut and who was unhappy about it.

What doesn't survive: the case study structure, the process narrative, the research summary, the reflection paragraph at the end. All of that can be generated, and none of it was ever the signal. It was a proxy for effort, and effort stopped being scarce.

In practice the portfolio stops being a filter and becomes a conversation starter. You still ask for it. You just stop deciding on it.

The follow-up sequence that settles it in four minutes

Pick one project from what they sent. Then ask these in order, and don't move on until each is answered concretely.

• “What was the constraint that isn't in here?” There's always one: a deadline, a legacy system, a person who wouldn't budge. Someone who was there names it immediately, usually with irritation. Someone who wasn't gives you a generality about scope.
• “What did you try first that didn't work?” Case studies present the final direction as if it were arrived at directly. Most projects have a discarded first attempt, and people remember them vividly.
• “Who disagreed with this, and what did they say?” The answer “nobody” is either false or describes a project with no stakes. Either way you've learned something.
• “If you had two more weeks, what would you have done?” This one separates by seniority rather than by authorship. A junior lists features; a senior names the thing they know is weak.

What this means for how you run the whole process

If the artefact is no longer evidence, the weight has to go somewhere, and there are only three places it can go.

Onto a live exchange: an hour where you can ask follow-ups. It's the most reliable and the least scalable, and it disadvantages people who interview badly and work well.

Onto an identical exercise everyone does under the same conditions. It's comparable, and vulnerable to the same generation problem unless it asks for a decision rather than an artefact.

Onto structured situations with no correct answer, assessed the same way for everyone. That's cheap and comparable, and it holds up because there's nothing to generate. Every option is defensible, so producing a polished version of one gives no advantage.

Most teams end up spreading the weight across all three. What almost nobody should still be doing is deciding a shortlist from portfolios alone, which is what most design hiring processes did in 2023 and many still do.

Frequently asked

Should I ask candidates to declare whether they used AI?

You can, but expect it to change little. Most people use it for writing assistance and will say so, and the ones you're worried about won't. A declaration policy mostly reassures the hiring team without changing what you learn.

Is it unfair to penalise AI use in a portfolio?

Penalising the writing is unfair. Plenty of good designers are bad writers, and always were. What matters is whether the judgment behind the work is theirs. That's a different question, and it's answerable.

Should I ask candidates whether they used AI?

Ask what they used it for. “Whether” invites a defensive yes or no and tells you nothing. “What for” gets you a description of their workflow, which is informative: someone using a model to pressure-test a decision works differently from someone using it to produce the artefact, and both will tell you if the question isn't an accusation.

Are AI-detection tools worth using on portfolios?

No. They produce false positives at rates that would be indefensible if you had to explain a rejection, and they fail on exactly the cases you care about. The same five minutes spent on a follow-up question get you a better answer, without the risk of rejecting someone for writing clearly.

Should I still ask for a portfolio at all?

Yes, for craft. Whether someone can hold a complex flow together is visible in the screens and hard to fake at length. The narrative around it no longer deserves your trust. Look at the work; interrogate the story.