.jpg) |
| Coffeepot for Masochists |
"No one wants to hear that their baby is ugly," stated Amy Rubino at the start of her class, Usability Testing for Everyone. She was referring to the fear that designers and decision makers sometimes have when it comes to usability testing.
My four classmates and I attended Amy's insightful three-hour lecture at the co-working space, Betamore. Her goal was simple: show us that usability testing is easy and necessary. With only five people in attendance, the class format was perfect for conversation - questions and stories flowed freely between the participants and Amy.
Below are some of my notes and takeaways from this fantastic class.
Below are some of my notes and takeaways from this fantastic class.
What makes something usable?
"When a product or service is truly usable, the user can do what he or she wants to do the way he or she expects to be able to do it - without hinderance, hesitation or questions" - Jeffery Rubin
Amy taught us that there are many considerations when it comes to deeming something "usable":
- Learnability
- Efficiency
- Memorability
- Errors
- Satisfaction (because even if it's usable, it still needs to enjoyable or else people won't come back)
How can we make a usable website?
Test it with your real users. These users should:
- Use your website
- Be part of your target audience
- Not be directly affiliated with your company
Through this type of testing, you are creating a user-focused design that is built from observed data, not the likes or dislikes of stakeholders. Amy let us know that in an ideal world, you would involve users throughout the design process - working iteratively by creating, testing and then repeating this process again and again. As designers and creators, we can choose to make this part of our creation culture - until it becomes second nature.
If you want to know the general principles for usability in interactive design, read Jakob Nielson's 10 Usability Heuristics and get the jist. In order to push beyond these basic principles, we need to make usability testing more tangible. One way of doing this is by creating personas. Who is your user, really? What does he or she need? What are her or his challenges? What technology is she or he using?
"Leverage any opportunity you can to listen to your users," stated Amy when advocating for guerrilla testing methods that she has found to be effective. As an example, she cited a project where her team visited with high schoolers during a job fair. There, she asked students to perform tasks on their readily available devices and had conversations with these real users "in the wild". Some of the results were eye opening. After testing, Amy and her team considered the fact that maybe teenagers aren't as tech savvy as we had assumed.
Common question: When can you test?
Answer: Anytime
- Early in development - Exploratory testing with paper prototypes or wireframes.
- Mid-way in development - Check points of usability.
- Late in development - Verification test before final execution.
Prototyping tools to try out
- Axure
- Justinmind
- Balsamiq
- PowerPoint (yes, even PowerPoint)
- Pen and paper
- Invision (I added this one to Amy's list!)
Categories of User Testing
In-Person Testing
Pros: Record conversation and click path, see users' reactions, involve your stakeholders and team
Cons: Users have to travel to meet, limits access to users who live outside of your region
Remote Testing
(Use screen sharing software to watch uses complete tasks)
(Use screen sharing software to watch uses complete tasks)
Pros: No travel, learn from people outside of your region, record conversations and click paths, involve your stakeholders
Cons: You can't see their faces/expressions
Unmoderated Testing
(Use a service that allows users to complete the tasks on their own time without you moderating)
(Use a service that allows users to complete the tasks on their own time without you moderating)
Pros: Users complete sessions when it's convenient for them, reach people outside your area
Cons: Can't ask followup questions, limited ability to hear the user or see reactions, need to watch videos for deeper analysis
Guerilla Testing
(Go to a public space and ask customers to complete a task.)
(Go to a public space and ask customers to complete a task.)
Pros: See and hear the users' reactions, view the website on different devices
Cons: Might need managers approval, customers might not be your audience, difficult to involve your stakeholders, difficult to record conversation and click path.
Putting us to the test
During the class, Amy had us create a preliminary test plan and lead a one-on-one usability testing session with a partner.
We took what Amy called a "streamlined approach". Here is the breakdown of our process:
1) Develop a test plan - this is your road map and structures your testing.
- What is your goal or purpose?
- Who are you targeting?
- Create a test screener for users
- What methodology will you use? Why?
- What are your metrics?
What are your goals?
Determine your objectives for reviewing this site. And be specific. For example:
- "Identify the obstacles in applying for a job"
- "Determine how quickly users can sign up for emails on the site"
Make sure you don't set people up for failure. Do a dry run with a colleague, and don't test major problems that will only cause frustration.
What are you performance metrics?
Goals example: Determine how quickly users can sign up for emails on the site.
Metrics example: Measure if users are able to sign up for emails within 2 minutes.
Other metrics include task completion rates (how many users are able to complete each task), time on task, page views, errors and measurement of satisfaction.
2) Prepare Test Materials
Script - This is what you will read to the participant before starting the testing. It will help to explain that you are not testing them, you're testing the website.
Tasks - Based on your goals, decide what activities your users should complete. These tasks must have an endpoint, but may also have multiple routes or "click paths". Be sure to ask, "Please let me know when you feel you have completed the task." Also, Amy says it's best to start with an easy task to build your participant's confidence.
Write each task as a clear statement in the user's language. Do not lead the participant with you questions. For example, if you want them to look for film studies courses on an educational site (and there is a button called "film studies"), simply state that they are someone interested in movies (not films) and they want to find related classes. That way the user is focused on the task - not the word "films".
Click Path - There should be at least one click path or route through the website for completing each task. Click path example: Home > Academics > Majors > Film Studies
Tips for Moderating
I loved this example she cited regarding careful and considerate moderating:
A participant had stated, "I would open Goggle and search for the website." In this test, Amy said that the moderator continued to call Google Goggle for the rest of the task in order to avoid correcting the user and disrupting their confidence. As a moderator, you're an anthropologist, not their buddy. But you also don't want to correct the user or react to much to their statements. Also, be aware that people want your approval, so they may hold back comments. This is where follow up questions can really work their magic.
Follow Up Questions
Ask open-ended questions that encourage conversation.
Bad: "Do you like the button?"
Good: "What are your thoughts on the button?"
- Stay neutral in your words
- Hold on answering the user's questions until the end of the session
- Don't take the feedback personally
Tips for Note taking
- Record whether the user completed a task successfully.
- What paths do they take?
- Where do they stumble?
- Do they understand what they are doing?
- "What's missing" "What would you change"
Focus on the user's behavior and comments. Write down direct quotes. Make sure to come up with a note taking strategy so that your notes are decipherable in post-test analysis.
Usability test example:
Common question: How many people do I need to test?
Answer: It depends, but 5 will usually do the trick
"The best results come from testing no more than five users and running as many small tests as you can afford" - Jakob Nielsen
For example, test with groups of five at six different points in the process. According to Steve Krug, five users will find 80% of your problems and three users find 65% of usability issues.
Try doing things in bite-sized portions. Test one thing a month. Slowly work on different sections of the website each time.
Common question: How do I recruit users?
Answers: Any way you can! Just make sure they are the right users.
There are services that can provide you with a target audience based on demographics - they basically pull from a pool of people connected with their services. But these people my be biased because of their previous experience with user testing. Also, friends and family always work great in a pinch!
Analyzing Data
- Review notes, audio and video and draw conclusions on the direction the project should take.
- Just because user said it, doesn't mean you should do it.
Takeaways
- Adapt usability - Start small, just plain start!
- Involve your stakeholders
- Test with a few people is better than no testing
"If you want to know whether your website... is easy enough to use, watch some people while they try to use it and note where they run into trouble - then fix it and test it again" Steve Krug from the book Don't Make Me Think
Usability Testing Tools
Screen sharing
Usability testing software
Eye tracking software
Unmoderated services
After the class ended, I felt motivated to weave usability testing into all aspects of my creative process. Not just for projects in my department, or for my freelance clients, but for any online interface that touches our users.
Amy, you may have created a test monster.
Amy, you may have created a test monster.
Convince Your Boss Tip: For every dollar you spend fixing issues in development, it costs $100 to fix the same problem after launch.
Recommended Books
Don't Make Me Think - Steve Krug
Rocket Surgery Made Easy - Steve Krug
Handbook of Usability Testing - Jeffrey Rubin and Dana Chisnell
