How to tell if a design portfolio was generated with AI

Stop trying to detect the text

AI-detection tools have a false positive problem serious enough that using them to reject a candidate is indefensible. Non-native English speakers and anyone who writes carefully get flagged constantly.

More importantly, the interesting case is not the fully fabricated portfolio. It is the real project written up by a model — where the work happened but the reasoning in the case study was generated afterwards. No detector can distinguish that from good editing, and it is by far the more common situation.

Six things a generated case study almost never has

Language models write the story a project should have had. Real projects have friction, and friction leaves specific traces.

• A constraint that was never resolved. Real projects ship with known problems. Generated ones resolve everything.
• A direction that was abandoned, and why. Models produce a single confident path.
• A disagreement with a named function — engineering said no, legal blocked it, sales overrode it.
• Numbers that are inconveniently precise. 'Activation went from 14% to 23%' reads differently from 'we significantly improved activation'.
• Something the designer got wrong and found out later.
• A decision made for a boring reason: no budget, no data, someone left.

The one question that settles it

Ask for a decision they now think was wrong, and how they found out.

It works because it cannot be answered from the artefact. Getting it wrong requires having had a belief, acted on it, and received contradicting information over time — three things a model summarising a finished project has no access to.

Someone who was there answers with a specific mechanism: a support ticket volume that went the wrong way, a metric that moved and then came back, a user who said something that did not fit. Someone who was not there answers in the abstract — 'I would have done more research upfront'.

Why this is getting harder, not easier

Two things are compounding. Portfolios get better because generation improves, and portfolios get more uniform because everyone is using similar tools on similar templates. The variance that used to carry signal is being flattened out.

The structural fix is not better detection. It is to stop relying on artefacts produced privately, in unlimited time, about work you cannot verify — and instead observe reasoning you generate yourself, under conditions you control.

The premise is wrong, and it is costing you candidates

Everything above assumes the goal is detection. It is worth stopping to ask whether it should be, because the detection framing produces two errors and only one of them is visible.

The visible error is letting through someone who cannot do the work. The invisible one is rejecting someone who can, because their writing was clean or their process diagrams were tidy. That happens constantly now, it leaves no trace, and it lands hardest on non-native English speakers and on anyone who used a tool to fix their grammar — which is close to everyone under thirty.

There is no detector, human or automated, that separates these reliably. Every published test of AI-text detection shows both false positives and false negatives at rates that would be unacceptable in any other part of a hiring process.

So the question is not «was this generated». It is «does this person have the judgment the work claims», and that question has always been answerable — it just was not necessary to ask before, because producing a plausible artefact used to require having done the work.

Assume everything was generated, and design for it

The cheapest reframe available: treat every submitted artefact as if a model produced it, and build a process that still works under that assumption. Not out of cynicism — it costs you nothing, because a portfolio made by a person survives this treatment easily.

What survives: anything that requires having been in the room. The constraint that was never written down. The reason the obvious approach was rejected. What the stakeholder actually said. What got cut and who was unhappy about it.

What does not survive: the case study structure, the process narrative, the research summary, the reflection paragraph at the end. All of that is generatable and none of it was ever the signal — it was a proxy for effort, and effort stopped being scarce.

The practical consequence is that the portfolio stops being a filter and becomes a conversation starter. You still ask for it. You just stop deciding on it.

The follow-up sequence that settles it in four minutes

Pick one project from what they sent. Then ask these in order, and do not move on until each is answered concretely.

• «What was the constraint that is not in here?» There is always one — a deadline, a legacy system, a person who would not budge. Someone who was there names it immediately and usually with irritation. Someone who was not gives you a generality about scope.
• «What did you try first that did not work?» Case studies present the final direction as if it were arrived at directly. Real projects have a discarded first attempt, and people remember them vividly.
• «Who disagreed with this, and what did they say?» The answer «nobody» is either false or describes a project with no stakes. Either way you have learned something.
• «If you had two more weeks, what would you have done?» This one separates by seniority rather than by authorship — a junior lists features, a senior names the thing they know is weak.

What this means for how you run the whole process

If the artefact is no longer evidence, the weight has to go somewhere, and there are only three places it can go.

Onto a live exchange — an hour where you can ask follow-ups. Most reliable, least scalable, and it disadvantages people who interview badly and work well.

Onto an identical exercise everyone does under the same conditions. Comparable, and vulnerable to the same generation problem unless what it asks for is a decision rather than an artefact.

Onto structured situations with no correct answer, scored the same way for everyone. Cheap and comparable, and the reason it holds up is that there is nothing to generate — every option is defensible, so producing a polished version of one is not an advantage.

Most teams end up distributing across all three. What almost nobody should still be doing is deciding a shortlist from portfolios alone, which is what most design hiring processes did in 2023 and many still do.

Frequently asked

Should I ask candidates to declare whether they used AI?

You can, but expect it to change little. Most people use it for writing assistance and will say so, and the ones you are worried about will not. A declaration policy mostly reassures the hiring team rather than changing what you learn.

Is it unfair to penalise AI use in a portfolio?

Penalising the writing is unfair — plenty of good designers are bad writers, and always were. What matters is whether the judgment behind the work is theirs. That is a different question, and it is answerable.

Should I ask candidates whether they used AI?

Ask what they used it for, not whether. «Whether» invites a defensive yes-or-no and tells you nothing; «what for» gets you a description of their workflow, which is genuinely informative — someone using a model to pressure-test a decision is working differently from someone using it to produce the artefact, and both will tell you if the question is not an accusation.

Are AI-detection tools worth using on portfolios?

No. They produce false positives at rates that would be indefensible if you had to explain a rejection, and they fail on exactly the cases you care about. Spending the same five minutes on a follow-up question gets you a better answer and does not risk rejecting someone for writing clearly.

Is it still worth asking for a portfolio at all?

Yes, for craft — whether someone can hold a complex flow together is visible in the screens and hard to fake at length. What is no longer worth trusting is the narrative around it. Look at the work; interrogate the story.