How to Recruit the Right Respondents for User Research?

Finding the right respondents for user research studies is no easy task. Typically, researchers have to attract interested participants, carefully vet them, then schedule a time to engage them. 

To complicate issues, at times, some respondents are not good candidates for the research as they cannot deliver meaningful feedback or insight. Unfortunately, suboptimal respondents negatively impact the quality of the study research.

What is Respondent Recruitment?

Respondent recruitment is a series of activities that focuses on recruiting the right respondents for focus groups, surveys, and in-depth interviews. 

What are the differences between participants and respondents?

In practice, participants and respondents are commonly used interchangeably, so people are referring to the same. However, if we are really particular about it, there is a slight distinction, a respondent is an individual who answers/responds to questions (either written or oral). While a participant is an individual who voluntarily joins to be part of a study as a subject.

Respondents mainly ‚Äėrespond‚Äô to the researcher‚Äôs structured and closed-ended questions. But, on the other hand, the participants go beyond simply responding to a series of questions.

For example, a participant elaborates on the researcher’s questions. Or can even change the topic if they want to convey an idea. As a result, research participants tend to provide more qualitative data than respondents.

Tips to consider when recruiting respondents for user research 

1. Understand your audience

Your respondents should suitably represent your target group. To achieve this, it’s essential to fully understand your audience.¬†

So, ask yourself: Who exactly are your users? Who do you seek to target? What are the actions or traits your users have? Then, based on the data you gather, you should be able to fully achieve the objective of your user research.

2. Create target groups

Respondents have different kinds of motivations, habits, and behaviours. In some instances, you need respondents who meet a specific requirement. 

For example, individuals who make food orders on a mobile phone instead of a computer. This can provide more comprehensive insight when surveying a food delivery service.

It is imperative to define what your exact criteria for respondents are. In essence, the choice of criteria typically depends on the particular requirements of your study.

This means that it’s also important to carefully create target groups in a manner that avoids underrepresentation of your market. 

target groups
Photo by Campaign Creators on Unsplash

3. Use screeners/ screening questions

Consider using a screener (screening questions) to sieve out the right respondents. You can achieve this by writing a short questionnaire that helps to determine whether testers are suitable. 

However, take note that long questionnaires might scare away your respondents. Remember, a single question can help you determine whether you have found a suitable respondent or not. For example, questions about the occupation, age, location, or education. 

4. Don’t include colleagues/family members

Research or testing colleagues or family members isn’t the best idea as human beings aren’t 100% rational. 

Colleagues and family members are more likely to have inherent biases and can get emotional. Therefore, will typically lack the objectivity to deliver actionable insight.

5. Pay attention to how you compensate

Simply put, money matters because respondents who feel fairly compensated rarely give much trouble. They don’t get grumpy about multiple phone calls, long screeners, or not qualifying at times. So, whenever they are eligible to participate, they are more engaged and excited. 

6. The recruiting process can deliver some actionable research insights. 

The recruitment process can reveal unexpected insights that could be useful. Because of this, researchers should always be open to learning and retooling the recruiting process accordingly, to get better insights.

7. Repeat respondents can sometimes be good. 

Individuals with good experience as respondents tend to be more reliable. Additionally, they understand that the process isn’t about seeking positive points of view, rather honest opinions based on real-life experiences. 

However, whether repeating a respondent will really depend on your user research objective. 

For example, repeated respondents will not be suitable if the objective is to validate the usability of a mobile app interface for first time users, since they have already seen the mobile app user interface in the previous study. 

How do I get respondents for user research?

1. Solicit website respondents

Similar to generating email leads, sourcing respondents on your website is a good idea. Visitors to your website can be politely asked to opt into a survey if they have extra time.

This can be achieved via a landing or squeeze page because the visitor has fewer decisions to make. However, on the web homepage, visitors have hundreds of distractions such as videos, or product demos. But a landing page isolates the call to action for your survey.

2. Pre-qualification calls

If you have collected a list of prospective respondents, the subsequent step is to determine whether they meet your criteria. 

Unfortunately, some behavioural qualities aren’t easy to determine over an online form. However, speaking to these individuals on the phone always takes it a step further and reveals more.

For example, you can determine whether an individual can articulate their thoughts well. Or even quickly ask their views on a general topic like ‚ÄĒ ‚ÄúIs Starbucks coffee overpriced?‚ÄĚ

3. Use a screener questionnaire

A screener questionnaire is a series of approximately 10‚Äď12 questions with different sequences of branching logic and termination points. The idea is that if, at any point during the screener call, the individual doesn‚Äôt qualify for your criteria. Then you must politely end the call.

However, specific things have to be handled while designing an effective screening questionnaire. The most critical point to remember is that you’re screening for behaviours, not demographics. Another vital point with screener questionnaires is to ensure that the recruitment criteria aren’t revealed to the potential respondent prior.

screener questionnaire
Photo by Alex Green from Pexels

4. Hire a recruitment agency

One way to avoid many recruitment hassles is to simply solicit the services of a recruitment firm. Most recruitment agencies have their own lead-time to recruitment services. 

But, remember to carefully review their screening question list to avoid mistakes that could lead to the wrong respondents for your research initiative.

5. Use social media channels

You can share your survey widely across different social media platforms to find respondents. 

Social sharing gives you a lot of visibility, depending on your topic. For example, your topic might suit a LinkedIn audience, since it’s dedicated to professional networking and job seekers. 

Or your topic might fit well with a Facebook group that regularly discusses Agriculture. So, it is imperative to use your discretion to decide which platform best serves your objectives.  

6. Utilise online panel vendors

Many recruitment vendors offer an online service to acquire participants. However, in some cases, their pricing could be high. If you decide to engage any, carefully check participants‚Äô demographics and psychographics within their databases to ensure it’s up-to-date.

7. Use dedicated panels

Relatedly, dedicated panels are fundamentally large databases of prospective research participants. Typically, for a fee, an organisation can screen and recruit the participants they need for your research. 

In some cases, dedicated online panels will deal with the logistics of paying out incentives. However, like most online panels, they often maintain many professional research participants. This can work against you if you want a variety and objective results.

So, ensure to utilise a well-crafted screener technique, and prepare to deal with participants who might not fully represent your target users.

8. In-person polling

This approach involves physically meeting your potential participants where they are. For example, if you’re looking at a specific target audience situated at particular locations, go meet them where they are. 

Especially, if you’re seeking feedback from a specific demographic less likely to respond online. For instance, demographics around a supermarket branded store or a physical location. Also, depending on participants’ qualifications, you can set up a table at a local trade fair, or conference to get respondents for your qualitative research.

9. Employ third-party channels and partners

If you have marketing partners, ask them if you can use their networks or marketing channels to contact potential survey participants. Most marketing resellers and suppliers will have access to multiple social media accounts and marketing systems with prospective participants.

How to get respondents for an online quantitative survey?

  1. Organic website traffic- depending on the nature of your study, you can host a survey invitation on your web homepage. Then allow prospective survey respondents to choose whether they qualify to take your survey.
  2. Random Device Engagement. 
  3. Use online community platforms like Reddit, Craigslist, QuestionPro Communities, etc.
  4. Share your survey on your website, social media, or blogs.
  5. Hire a dedicated Market Research agency. 
  6. Utilise your existing customers’ database since they already have first-hand experience using your platform. 
  7. Send surveys via email campaigns
community platforms
Photo by Brett Jordan on Unsplash

Conclusion

The success of user research depends on its respondents. More precisely, on one’s choice of respondents. This means that finding the right participants is vital to extracting valuable results from your user research. 

Remember that participants have to be able to fairly represent your end-users. So, carefully consider the criteria you have for your research participants¬†before¬†commencing recruitment. For example, when your criteria are quite general ‚ÄĒ like age and location ‚ÄĒ recruiting participants is considerably easy.¬†

However, when you have more particular requirements for candidates to participate in your study, it can be very difficult and time-consuming. Though by balancing extreme and mainstream users, you can improve your chances of getting objective and appreciable research outcomes.

How To Conduct Usability Testing For Websites?

Theoretically, an ideal website should be immediately clear to new visitors, easy to use (even after long breaks from the site) and error-free. All these while delivering a pleasant experience for all users. 

While this might seem obvious, in practice, it’s quite challenging for many organisations to achieve. Unfortunately, very few web interfaces are perfect as many websites are typically poorly laid-out, therefore making user navigation counter-intuitive. 

For instance, their web components comprise poorly labelled buttons, or unclear prompts, which frustrate new or returning users. One way to avert such problems is usability testing. 

Usability testing for websites

What is usability testing?

To accurately define usability testing, we need to first define usability. Usability can be generally defined as the degree to which a device or software assists a person to accomplish a specific task. This is in contrast to becoming an additional impediment to the task’s accomplishment. 

That being said, usability testing is a set of non-functional testing techniques employed to measure how easily a software system or hardware device can be utilised by end-users.

What is website usability?

Website usability testing, also sometimes referred to as User Experience (UX) testing or user testing, involves using testing techniques to measure how easy, intuitive, and user-friendly a software application is. 

Website usability test revolves around exposing usability defects by determining users’ ease of using an application, the flexibility of the controls, and the capacity of the application to meet its objectives. 

The human-computer interaction characteristics of software are measured, and weaknesses are identified for correction.

As such, usability refers particularly to how well people engage with a website.

Jakob Nielsen, probably the most famous usability expert, believes that usability constitutes five key components: learnability, efficiency, memorability, minimisation of errors, and satisfaction. These can be broken down as: 

  • Learnability: How easy is it for website users to accomplish basic tasks the first time on the site.
  • Satisfaction: How pleasant and easy is it to use the website?
  • Efficiency: Once visitors have learned the design, how quickly can they perform tasks?
  • Memorability: How easily can return users re-establish proficiency?¬†
  • Error rates: How many, severe and permanent are user errors?¬†

Why is website usability testing needed?

Usability tests are typically conducted for multiple reasons, namely:

  • To determine the time it takes to complete a task compared against established benchmarks.
  • To ascertain user satisfaction by understanding user pain points to come up with applicable design solutions to improve user performance.
  • To evaluate if users can navigate your website.¬†
  • To identify potential problems with website functionality.
  • To establish if a website is accomplishing an organisation‚Äôs goals.

Overall, the primary goal of usability testing is to ensure that people can utilise your website.

Website usability testing methods

There are numerous methods and techniques for usability testing, namely:

Comparative Usability Testing

In comparative usability testing, end users are essentially required to choose the best of two or more options. It’s principally performed to check the effectiveness of a website against that of its competitors to identify any shortcomings, then address them.

It involves observing metrics like error rates, task completion, and time spent on each specific task.

Explorative Usability Testing

Unlike comparative tests, explorative usability tests are more open-ended as participants are encouraged to give their opinions and explain how they feel about a specific design or user flow.

It is usually performed during the initial stages of website development to judge how users might react to the design. This is to create a fully informed process with users’ exact needs in mind.

Cognitive Walkthrough

This testing approach is designed to assess how a website interface supports first-time users while they learn how to complete a task. 

The designers and developers basically imagine the possible steps that would be performed by a user to complete a particular task. They then evaluate the website system’s responses to those tasks. 

Cognitive Walkthrough
Photo by UX Indonesia on Unsplash

Focus Groups 

Here, a small group of people (participants) sit around a table and interactively react to ideas and designs that are shown to them.

Heuristic Evaluation

In this testing approach, testers examine a website to judge how well it conforms to standard usability principles. Evaluators systematically compare a website against a set-list of defined criteria to determine how closely its design follows recognised usability principles.

Think Aloud Testing

The thinking aloud approach entails observing test users’ interactions with the website as they navigate through the website, and vocalise their thoughts.

In practice, users complete a particular task, vocalising their thoughts as they navigate while explaining why they performed each action and what they are looking for. As such, anything that seems confusing or frustrating is marked upon and recorded for action.

Remote Usability Testing

In this testing approach, the testers and users could be in different countries and time zones. As such, remote testing is, at times, performed using video conferencing software, while other times, the website user works separately from the evaluator. 

Currently, remote testing software allows remote usability testing to be performed even by observers who are not usability experts. Usually, this software automatically records the click locations and streams of the users. 

As well as any critical incidents that happened while they were exploring the website, along with any actionable feedback the user submits.

Guerrilla Testing

The guerilla testing approach of a usability test is probably the cheapest and simplest of the lot. In practice, it involves gathering feedback from random people.  

Sometimes, this method involves paper prototyping as an alternative to creating a functional prototype website. Overall, guerilla testing is a time-efficient technique since you do not have to recruit qualified or vetted participants, then wait for their responses.

Read more in our article on ‚ÄúGuerrilla testing‚ÄĚ

Phone Interview Testing

As a usability testing approach, phone interviews involve a researcher asking questions to testing participants. It could also be done by instructing them on how to perform a specific task over the phone. In turn, the participants give feedback regarding the website in question.

Overall, this technique is helpful when reaching out to participants and gathering data from wider geographic areas. This is because it allows you to obtain a broader perspective of the website’s potential issues.

Steps to conduct website usability testing

  1. Clearly determine metrics and create task analysis

It is imperative to figure out the precise metrics you‚Äôll consider during your testing. Since usability testing can uncover a host of issues, if it‚Äôs not being targeted around specific metrics, it won’t be an effective use of your time or money.¬†

As such, ensure to clearly state what you seek to achieve with the usability testing and the exact information you hope to gather. For instance, if you intend to know if website users can order a product successfully, then you should test the entire process from the front page to completing the order. 

Overall, usability metrics play a crucial role as statistics to measure a user’s performance on a given set of tasks. For example:¬†

  • Success Rate: This judges whether the user was able to complete the task.¬†
  • Error Rate: This metric looks at the errors tripped up by users most. They can be divided into either: critical and noncritical. In practice, critical errors prevent a user from completing a task. While noncritical errors lower the efficiency with which they complete tasks.
  • Time to Completion: This measures the amount of time a user takes to complete a task.¬†¬†
  • Subjective Measures: This metric numerically ranks a user‚Äôs self-determined satisfaction, ease-of-use, information availability, etc.
success rate
Photo by krakenimages on Unsplash
  1. Identify the best applicable test method

Usability testing takes multiple forms that come in a range of difficulty and investment requirements. That being said, it is important to determine the best type of test for your website depending on the metrics and tasks you have curated in step one.

After you set the test’s goal, decide which usability testing method is the most suitable. Also, consider the resources you have to do a website usability test.

  1. Create task scenarios and set your success rate

Ensure to create different task scenarios that participants should accomplish. For example, a series of tasks related to the aforementioned product ordering example we’ve used in step two. 

  1. Find suitable participants

From the outset, it is important to carefully determine the number of users you need to conduct a satisfactory usability test. Typically, the recommended industry standard is a minimum of five users.

As you source users, ensure that they are fair representations and approximations to real users. In practice, not validating website design changes with a unique user base can have some drastic implications to your site’s usability. 

  1. Decide when, where, and who

Essentially, this step involves choosing between two key decisions: 

  • Remote or in-person testing?
  • Moderated or unmoderated testing?

Website usability testing checklist

As you embark on your usability testing initiative, ensure that you check this checklist to increase your chances of success: 

  • Written website goals.
  • Written tasks users take to accomplish goals.¬†
  • Written scenarios and situations in which users would engage in these tasks.
  • A list of the usability metrics you will capture during the test.
  • A list of selected test methods.¬†
  • What point(s) during your project will your tests take place?
  • Number of testers you will need if a moderator is involved.
  • If you are conducting an online test or a paper test?¬†

Conclusion

Usability testing is an absolutely necessary activity for any business with an online presence.  As we have established, the three key metrics that are important for usability testing are satisfaction, efficiency, and effectiveness. 

Fundamentally, these metrics will determine the best type of usability test to perform for your website.

That being said, usability testing should be more than checking a list of product requirements, but an exercise to support your website design decisions.

Overall, by listening to users and understanding how they interact with your website, you can avoid spending unnecessary time and resources. And, in turn, better serve your users. Remember, usability testing should principally determine whether a website is:

  • Useful
  • Findable
  • Accessible
  • Usable
  • Desirable

The article is a part of our comprehensive series on ‚ÄúUsability testing‚ÄĚ.

What Is Guerrilla Testing and How to Use It?

More often than not, new products like electronic appliances, websites and mobile apps can cause users frustration as they may be tedious to use, lack flexibility, and take attention away from their core purpose. 

Today’s topic focuses on one of the techniques to mitigate such scenarios by employing guerrilla usability testing.

What is Guerrilla Testing?

In essence, guerrilla testing is a usability testing technique that involves gathering user feedback. It is accomplished by taking a product design or prototype to the public and asking random passersby for their thoughts. 

Typically, guerrilla testing sessions last for only 10-15 minutes, with a small incentive given to users like a coffee, coupon or cake.

Because of its simplicity, new ideas can be quickly tested at a low cost. As such, guerrilla usability testing is considered an inexpensive means of testing mobile apps, product prototypes or websites with real-life users. 

Guerrilla Testing Deliverables

For the most part, guerrilla¬†testing¬†deliverables are typically more qualitative than quantitative as they involve direct assessment of participants.¬†In practice, qualitative approaches tend to query ‚Äėwhy‚Äô rather than ‚Äėhow many‚Äô or ‚Äėhow much‚Äô.¬†

As a result, guerrilla testing seeks to inform development and design decisions for ongoing projects to identify usability issues, rather than assess the overall usability of an existing product or interface.

Thus, guerrilla testing is ideal for:

  • Getting fast baseline measures of an existing product experience.
  • Identifying crucial usability issues early in a product design lifecycle.
  • Testing hypotheses and validating assumptions during design sprints.¬†
  • Validating tasks that do not necessitate specific knowledge (for example, completing a signup form, or ordering a product in an e-commerce store)

Guerilla Testing vs Usability Testing

Usability testing revolves around testing how easy a particular design is to use with a specific group of representative users. In practice, it traditionally entails observing users’ reactions and behaviours to a website, app or product.

The group of users attempt to complete tasks at different stages, from early development until a product’s release. It is carried out in a controlled environment, such as a lab or designated room or online 1-to-1 session. 

Guerrilla testing is considered a subset of usability testing and takes a more agile approach to testing a prototype, product or website. High-level feedback is derived to find and fix potential UX issues. Guerrilla testing can also be performed at various stages in the project’s life cycle. 

Benefits of Guerrilla Testing

Guerrilla testing offers numerous advantages to product developers, such as:

  • Quick¬†turnaround as there is no waiting around for recruiters to find people with specific qualifications or attributes.¬†
  • Inexpensive compared to formal testing as there are no travel costs for users.
  • It enables testers to identify any UX barriers¬†early in the development process.
  • It is iterative and works well with an agile project approach.
  • It provides sufficient or enough insights to inform strategic design decisions.¬†
  • Guerrilla research is flexible and can be squeezed into nearly every timetable or deadline.
  • It can be utilized to demonstrate the value of user testing/research for stakeholders, especially for those who may struggle to acknowledge the value of usability testing.
  • It is a great way to do ad hoc user research, whether when conducting competitor analysis for similar ideas or practising moderation skills.¬†
  • It delivers substantial ‚Äúcontext of use‚ÄĚ observations.¬†
  • It provides a lot of ‚Äúusefulness‚ÄĚ feedback in real-life situations.¬†

Shortcomings of Guerrilla Testing

There are, however, some disadvantages that should also be kept in mind:

  • You may not get the right target audience.
  • It could be challenging to record feedback.
  • Sessions are short, so one could lose some of the insights that they would typically get from formal user testing.
  • It may not be really appropriate for all types of websites or mobile apps.
short discussion
Photo by Headway on Unsplash

How to Conduct Guerrilla Testing?

In contrast to recruiting a particular targeted audience to take part in testing sessions, with guerrilla testing, participants are usually approached in public locations and asked to take part.

Since there is no formal recruitment or requirement for expensive research facilities, guerrilla testing sessions are fast and easy to set up and can be conducted anywhere like a coffee shop, library, park. 

It is recommended to test between 6 -12 users, though this can vary depending on who and where you are testing from. Guerrilla testing sessions are typically short (10‚Äď15 minutes) and are structured around specific key research objectives.¬†

If you plan to record the sessions, it is advisable to get participant consent.

Each guerrilla testing session can be initiated with the following number of steps:

  • Approach a potential participant.¬†
  • Politely introduce yourself to potential participants and ask if they would like to partake in the software or product testing session.
  • If they agree, get general information about them.
  • Have them sign a consent form.¬†
  • Give them a few different scenarios.
  • Carefully observe their interactions.
  • Ask about their experience with the product.
  • Thank and reward them for participating.

Tips for Guerrilla Testing 

Following are some tips to help you achieve your testing objectives:

  • Carefully think about all the critical things people need to be able to achieve while using your product, and write down a shortlist of tasks. For instance, if your product is a mobile app for ordering food, you‚Äôll want to test how people find a particular meal, order multiple meals, or add a meal to their cart.
  • After listing your tasks, prioritise them and decide what to exactly test. Then choose the top 3 tasks, and use them to create scenarios users can easily comprehend.
  • Ensure to always create an elaborate scenario based on each task. In essence, a good scenario describes a problem for participants to solve and is relatable, but does not hint to the participant how to achieve the ultimate goal.
  • Before testing scenarios with test participants, ensure to pre-test them with friends and colleagues to ensure that people can follow them without any confusion.¬†
  • Always be ethical and transparent.
  • Understand people‚Äôs expectations even if your design is not fully complete. For instance, use this opportunity to ask them what they might expect to see after clicking on a certain button/link in a User Interface (UI) flow¬†
  • Because guerrilla testing is meant to be an ad hoc technique, it does not mean it should be entirely unplanned. So, make sure you carefully plan how much time you‚Äôre going to ask of users.
  • Always stay polite and ask users if they are okay with you stealing a little more time than planned. Generally, people won‚Äôt be happy if you don‚Äôt respect their time, which may lead to¬†biased or even unhelpful responses¬†to your questions.¬†
  • Avoid always conducting your guerrilla testing in the same places or types of places. Testing at different places will help ensure that you have participants with varied demographics.¬†
ethical
Photo by Nathan Dumlao on Unsplash

Guerrilla Testing Questions

As we have established, guerrilla testing is a low-cost usability testing method that helps you quickly answer straightforward usability questions. That being said, guerrilla testing questions vastly differ from those employed in more structured and in-depth research methods.

Principally, guerrilla testing questions are shallower in their depth of insight, and seek to be informal when engaging users. 

What should you not test?

Guerrilla testing should not be employed in specific scenarios like:

  • When domain-specific knowledge is necessary to use a product (for example, when completing specific use-cases in financial or even medical apps). In practice, you cannot expect random people to have all the required skills to deliver satisfactory technical answers.
  • When a highly specific environment is necessitated to conduct testing (for instance, when testing can be done only in a specific location).

Guerrilla Testing Example

Overall, where you conduct guerrilla tests affects how you perform and document your work. For example, if you’re testing a new mobile app for a supermarket chain, you might go to the store itself and walk down the aisles. 

However, if you’re working on a “general” office software suite, you might test it with workers in a different part of an office, etc.¬†

Overall, the idea is to let context drive your decision making. Though public spaces and shopping malls may present some of the best locations for guerrilla testing due to the sheer amount of free foot traffic they receive. Not to mention the relaxed nature of the environment, which can come in handy when approaching strangers.

However, with more particular user sets, you can target subjects based on context and demographics. Furthermore, companies can perform remote guerrilla testing by exploiting public forums such as Reddit, Quora or even LinkedIn Groups, by simply writing a simple post describing the intent and the related incentive.

Takeaway

In conclusion, if you’re seeking quick feedback on your prototype or website, then guerrilla usability tests are the way to go. Guerrilla testing allows you to conduct multiple expedited field usability tests during the course of project development to gain actionable insights into where usability barriers could be.

However, you have to be considerate of the occasional risk of not getting in front of the right target audience, and sessions being much shorter than with formal usability testing. This might limit your ability to get comprehensive insight. 

Nonetheless, guerrilla testing gets you in front of all user types; whether it’s a prospective user, or Peter from down the road, who opens every app under the sun, apart from yours. 

Overall, guerrilla testing is a great way to get your foot in the door to demonstrate the value of spending time with users before launching any final product.

Reach out to us at Netizen Experience to discuss your usability testing needs.