In The Hitchhiker’s Guide to the Galaxy, an experiment to find the meaning of life yields a famous answer that cannot be understood without asking: Uh, what exactly was the question?
Clarity is critical in user testing. Testing sessions often include pointed questions like: “If you wanted to change the language to French, how would you do that?” Testers are wondering if the user will think to look in a certain place for a way to complete the task. Here is a problem, they are saying. Solve it.
However, giving a user a problem to solve is less useful than outlining a scenario and letting them frame the problem themselves.
From the perspective of the user, when asked for an answer, she must first be sure she understands the question. This happens on different levels. Subconsciously she might say to herself: But I don’t speak French. Why should I change the language? Consciously she might wonder: Is the solution somewhere unlikely? What is the question getting at?
The example above is very simple, but as tasks become more complex, understanding the syntax and intent of the question becomes a problem in itself. It introduces a level of uncertainty that clouds the purity of the test: even if an answer comes readily to mind, she must then reconsider her understanding of the question, to see if the two match. Solving someone else’s problem is much less intuitive than solving your own.
If the tester frames the question as a scenario, rather than a problem, that uncertainty evaporates. “The website is in English, but you speak French. What will you do?”
The difference seems trivial, but in user experience trivial things matter. Now the user can frame her own problem (I need to change the language) and then begin working on the solution. She already knows she understands the problem, because she articulated it. This approach yields more authentic test results, since in the real world we don’t have annoying UX testers asking us leading questions. We identify, and solve, our own challenges.