Assumptions begin in the conceptual stage, well before design
— How can we test our assumptions to avoid design pitfalls?
We have, on our product team, the most important player: the folks who are using our product. Then we have our developers, our architects, and ourselves. Every team member carries different pairs of lenses with its own underlying assumptions.
Dr. Laura Faulkner presents a project
- Our client desired a SaaS platform to organize file systems.
- The platform has sharing capabilities between the admin and their team.
- The admin of a particular file system granting permission to another team so that the team can also share with other teams within the company.
The problem
> We (the designers) felt that users won’t adopt the solution.
The hypothesis
> If we build a prototype and invite our users to walk through it, they will resist sharing permissions further along the chain!
The result
> People were okay with it.
We were wrong!
How can that be — did our intuition fail us?
Trust your intuition and keep investigating
What was missing in our test?
- Did we test the wrong group of people?
- Did we write the test plan wrong?
- Was our hypothesis wrong?
Let’s examine our test plan with the whiteboard
Begin with our users:
- The owner of the file system
- The colleague they want to share their file system with
Think about user goals:
- The owner shares their file system with their colleague
- Their colleague wants to share the same file system with a colleague on a different team
- The other colleague wants to share with another person…
- And so on, you get the picture
Key finding:
At the third share, people began to get uncomfortable. The owner of the file system felt their work was getting out of hand. Trust was breaking down.
How did this happen?
The experience design was so good that people made assumptions. They skipped right past the idea that there might be a security risk in the model.
Key learning:
Visual design can interfere with interaction design in this case — therefore, incorporate visual design last.
How can we improve the test plan?
- Start with business goals. What metric do we want to improve?
- Move onto research goals. What do we aim to learn through research. In this scenario, we want to discover how people feel about sharing without permissions.
- Now, we can draft our objectives, which are questions we want answered. In this case, we want to know how many people down the chain are people willing to give sharing permissions to.
Once we define the previous sections, we can move onto drafting the method, questions to ask, and script. A storyboard is a great way to put people into the scenario, out of the tasks they are doing with the interface.
Key findings
- People were not uncomfortable with the ability to grant permissions but…
- They were in the blind of who those permissions were granted to down the line
Solution
- Make the logic of permissions sharing transparent
Outcome
- The solution gained people’s trust
Design your own abstract model test
- Trust your own doubts and those of others
- Imagine the disaster
- Disasters happen when latent conditions align
- Write disciplined test goals and objectives
- Don’t plan to show the participant the actual design or prototype
- Test experts as well as novices
- Develop a story to place participants in
- Create a visual storyboard to pull participants in
- Question the participant at key points — ask how they feel
- Develop a model from the results of your research
Tip: be gentle with yourself and others!
Wrapping up
Recommended reading:
- Managing the Risks of Organizational Accidents by James Reason
- Blink by Malcolm Gladwell
- Beyond the five-user assumption: Benefits of increased sample sizes in usability testing by Laura Faulkner
Recommended actions:
- Share resources/insights to stakeholders and offer to do research on it
- If you’re starting out in UX research, do a heuristic evaluation on something you hate
