Last Updated on July 26, 2020
After conducted thousands of usability test sessions with clients and partners in the past many years, we still find that people asking the same question, for example:
- Is usability testing same as UAT (user acceptance test)?
- Do I need 100 or 500 users for usability testing?
Hence, we wish to clarify the doubts that most people have about usability testing by answering few key questions in this article:
- What is Usability Testing?
- Why Usability Testing?
- How to conduct Usability Testing?
- Who? How many users? How many testing cycles?
Usability Testing (some prefer to call it User Testing or UX Testing) is a crucial element part of UX (User Experience) design. As the name suggests, usability testing is a common technique used by usability professionals and UX practitioners to evaluate the usability of a product by testing it with real users.
It is totally different from User Acceptance Test (UAT). UAT is the final phase of the software testing process. Typically, software users are tasked to test the software to make sure it can handle required tasks in real-world scenarios, and according to specifications.
Although usability testing is favoured by developers and designers, such methodology & skillset is now widely adopted by a greater range of professionals and individuals from different fields such as:
- research & development
- digital strategy
- customer service
Test it with real users
When we say, ‘test it with real users’, the message is clear – conducting usability testing is totally different from conducting a technical testing which is another crucial process in software development. Usability testing focuses on the real user, to study their interaction with the product such as digital touch points, websites, mobile apps and even tangible products.
To put it simply, Usability Testing is one great tool to evaluate the user experience of your products. It provides inputs to the question, “How user-friendly is the product”. Nonetheless, usability testing allows you to discover opportunities to improve user experience as well as conversions.
We’re now living in the era of digital products and information. With new mobile applications and websites being launched every day, delivering a user-friendly product that emphasizes a user’s experience puts you ahead of the competition.
Usability Testing is a qualitative type of research which serves as a great compliment to quantitative research & testing tools like market survey, web traffic analytics, A/B testing & multivariate testing.
Testing helps to evaluate and ensure that you’re delivering the right experience to users. Through usability testing, we are able to track not only usability issues, but also many other aspects including content requirement, user journey and preferences.
Remember, testing not only helps you to track what is wrong with your product, it increases your confidence level when a feature or a great element of your product receives approval or is commended..
As mentioned above, usability testing is no longer exclusive to only designers and developers. We have seen marketing agencies that utilize usability testing to study user behaviour and preference on their own products, a competitors’ product or any other products that they’re interested to venture in.
To a certain extent, individuals (e.g., bloggers) also conduct testing to test the UX of their website to study how blog visitors behave while browsing content and advertisement.
We often hear marketing teams say that they conduct ‘similar’ testing on their digital products, whereby they exposed their products to customers or potential customers to enquire their impression of the products without getting them to actually use the products.
Impression tests are great but can be misleading when it comes to evaluating the utility and user friendliness of products. Consider, for example, when a market research firm provide reports for a proposed product that receives a high score or rating from users based on their impression of the products:
The customers were exposed to the product at a glance and were asked questions like, “Do you think you can subscribe to this service easily?”
“Sure thing, I saw the subscription button right here”.– The end.
Following this, when it comes to getting the users to actually go through the subscription process after clicking the ‘subscription’ button, the registration and navigation process turns out to be rather tortures.
This is another time when usability testing can fit in to clear doubts.
The commonly known way of conducting usability testing is through laboratory tests where users are invited to a lab to test a product. Generally, there are 3 main roles in a lab test session:
- the user (or tester),
- the moderator
- and the observer.
The user will be the one to test the product by performing certain tasks that are laid out for him/her.
The moderator will be the one who moderates the test session by providing task guidance to the user and will ask questions about the product during the test. One precautionary measure for moderators is to ensure that questions asked are non-leading questions, for example, instead of directing the testers to the checkout process step-by-step, it would be more appropriate to ask them to figure it out by themselves.
The observer usually remains silent and observes how the user interacts with the product, takeing notes.
The skillset of asking the right questions in the right way at the right time is not easily mastered; but one will always get better when practicing more in moderating the test session.
Similarly, note-taking & analysis can be challenging for those who are not familiar with the process. Observers should focus on recording down as much detail as possible to avoid missing any insight for subsequent analysis.
Sometimes, it involves 2 or more observers that might comprise of project managers, product owners, designers or other stakeholders. To prevent losing any insight from the testing session, the testing session can be recorded with a video camera so that the team can always review the testing session.
The key requirement to propel an insightful usability testing session is the existence of a ‘think-aloud protocol’. Users are encouraged to express verbally what they are thinking throughout the whole session, so that observers and moderators can get greater understanding on a user’s experience when interacting with the product.
From here, moderators can also probe for further comprehension should there be any uncertainty. Setting up a full-fledged usability lab can be very costly as it requires certain advanced equipment, and hence, some practitioners still opt to conduct usability testing with minimal equipment such as a laptop & a recording device.
There are many opting to conduct Guerrilla Test. Guerrilla Test totally opposes traditional lab tests as it is ‘hunting in the wild’ at public places such as parks & cafes where you can easily access the mass public.
However, there are also many who are reluctant to conduct both lab tests and guerrilla tests because these can be very costly & time consuming, considering that it might not be easy getting the right candidates in a short time. Another good alternative is to conduct remote usability testing, which is proved to be quick and cost-saving.
For a remote testing session, a moderator is not required to be physically present beside the tester. Remote usability testing can be moderated, or un-moderated: which means that there is no one moderating the test session.
If you are keen to learn more about remote usability testing and its effectiveness, we have elaborated more in this article, and also shared comparison result of remote usability testing and classic lab testing.
You should not use your own team members or your friends & family to test your products, as there is a high possibility of getting false results. Your friends and family might tell you ‘sweet’ comments to fawn you instead of telling the exact truth whilst your team members are usually as passionate as you in believing in the product.
Whereas for users who have no direct relationship to you have the freedom to speak out their mind and they react exactly how an ordinary user would without any conflict of interest.
A more challenging situation is that it might be difficult to get users who fit your criteria to test your product. For example, if you’re an education service platform provider in Malaysia aiming to provide services to children in Indonesia; it can be costly and time consuming to fly in to recruit the right participants for the test.
The next best alternative would be getting the local users (in your country) to do testing, provided that they must fit the crucial criteria of your target users such as age group, gender, education level and computer skill.
The most frequently asked question would be “how many testers do I need to test? 50? 100?”. Believe it or not, you need only 5 testers (to start or try).
To further explain the logic on that, here’s a good explanation by experts on why you only need to test with 5 users. The below graphs illustrates that by using 5 users, you are likely to obtain 85% of the usability problems of your product.
A Research Graph by Nielsen Norman Group
Just like the law of diminishing marginal utility in economics, the first tester generally yields more (newly-identified) usability problems than the second & subsequent testers. Less or minor usability problems would be found after the sixth tester. Our experience in conducting hundreds of usability test session has proved this point too.
If you have the budget for conducting a test with 30 users, it is recommended separating the test to two (15 users each) or three sessions (10 users each) depending on your development phase and requirement.
To summarize, it is a qualitative research method which does not require huge number of sample size as conducting a market survey research. You do not need 100 users to tell you that a button or link is not working.