Common Mistakes When Testing Prototypes with Real Users

Watch a prototype test go wrong and it rarely looks dramatic. Nobody shouts or storms out. The participant smiles, clicks roughly where they are pointed, says the flow is "really clear", and everyone leaves feeling good about a design that will collapse the moment it meets a real task and a real deadline. The mistakes that cost the most are quiet ones, and most of them start before the first participant sits down.

Asking questions that contain the answer

"How easy was that to use?" sounds harmless. It is not. It tells the participant that ease is the expected answer, and most people would rather be agreeable than accurate. You get a nod, you write down "easy", and you learn nothing.

The same goes for "Would you use this?", which asks people to predict their own future behaviour, and for double-barrelled questions such as "Was the pricing clear and was it fair?", where the participant answers whichever half they happen to remember.

Move from opinions to behaviour. Ask about the past and the present, not the imagined future, and ask people to show you rather than tell you.

Instead of: "Was that easy to find?"

Try: "Show me where you would go to change your delivery address, and talk me through what you are seeing."

Then stop talking. The most useful sentence in a research session is often the one you do not say. Count to five in your head before filling a silence; participants frequently correct themselves, or reveal the exact assumption you needed to hear.

Polishing the prototype until it lies

Fidelity is a dial, and it should match the question you are asking. A prototype with finished copy and real-looking data invites a different conversation from one made of grey boxes and placeholder text.

Over-polished work tends to produce two problems. First, participants become polite critics: they comment on a button label or a shade of blue because those are the things that look finished and therefore feel fair to question. Second, they forgive broken interactions because it all looks so convincing, and you lose the honest friction you were hoping to observe.

Rough work has its own failure mode. If a participant cannot tell what a screen is roughly for, you are measuring their ability to read a wireframe rather than your idea.

• Testing a concept or flow? Keep it low fidelity, focus on structure and language, and expect the visuals to be ignored.
• Testing one specific interaction? Make that interaction work properly and leave the rest knowingly rough.
• Testing whether it feels trustworthy? Then craft matters — but say so, and ask about trust directly.

Whichever you choose, tell participants what stage the work is at, and make it clear that blunt reactions are useful rather than rude.

Treating accessibility as a later problem

If every session involves a mouse, a large screen and a quiet room, you have tested a narrow slice of how the thing will actually be used. Accessibility feedback is design feedback, and it is far cheaper to hear early than to retrofit.

Practical ways to widen the net:

1. Run at least some sessions with keyboard only, a screen reader, voice control, magnification or large text settings in use.
2. Ask about workarounds. If someone keeps browser zoom at 150 per cent, or copies text into a notes app to read it comfortably, that is a finding.
3. Test in the conditions people really use — on a phone, one-handed, on a train, in daylight, with notifications arriving.
4. Pay participants for their time and expertise. Do not treat disabled users as a free quality-check service, and do not ask them to diagnose the cause of a problem for you.

Some sectors carry formal accessibility obligations. If you are unsure what applies to your organisation, take proper advice rather than working it out from a blog post.

Recruiting whoever is nearest

Colleagues, friends and whoever happened to be in the office are the easiest participants to book and the least useful. They know too much, they want to be encouraging, and they have absorbed your vocabulary.

Recruit for behaviour rather than job title. "Has booked a short break online in the last six months and abandoned at least one booking" is a usable brief; "25–40, likes travel" is not. Write a short screening questionnaire, keep it to a handful of questions, and be willing to turn people away.

Work in small rounds rather than one enormous study. A handful of participants, then a fix, then another round, teaches you more than a dozen sessions that finish after the design has already moved on.

Mistaking requests for requirements

Participants are excellent at describing problems and unreliable at prescribing solutions. When someone says "you should add a filter here", the finding is rarely "add a filter". It is more likely "I cannot tell which of these items are relevant to me", or "there is too much on this screen".

Keep a light structure in the room so the evidence stays usable: one person moderates, another notes the timestamp, the task, what the participant did and what they said. Capture the moment of hesitation, the misread label, the second attempt. Those details survive into a design decision; a general impression of the session does not.

Resist explaining the interface when someone gets stuck. Rescuing a participant feels kind and destroys the finding. Note it, let them struggle for a moment, then move on.

Before your next session

Half an hour of preparation protects the whole round. Before the first participant arrives, check that your tasks describe a goal rather than a route through the interface, that your questions cannot be answered with a simple yes, and that at least one session includes assistive technology or an unusual setup.

Afterwards, resist the urge to jump straight to solutions. Write down what you saw, group the observations, and only then decide what to change. The point of testing with real people is not to collect approval. It is to find out, cheaply and early, where your assumptions were wrong.

Photo: Firmbee / Pixabay