Tips For Conducting Usability Studies With Participants With Disabilities

Over the last few years, I ran several usability studies with participants with various disabilities. I thought it would help others if I shared some of my experiences.

In this article, I provide lessons learned or tips to consider in planning and executing usability testing with participants with disabilities. The lessons learned are divided into general that can apply to all types of disabilities; and lessons learned for three specific disability categories: visual, motor, and cognitive. These tips will help you regardless where you work: If you work with an established user research team where usability testing is part of your design process or if you work on your own with limited resources but want to improve the quality of the user research by expanding the diversity of participants.


Windows 10 high contrast mode of Google.com
Windows 10 high contrast mode of Google.com. (Large preview)

Background

Several of our clients from a state government agency to several fortune 500 companies came to us at the User Experience Center (UXC) for help with their websites. They wanted to make sure users with disabilities could access their site and accomplish their goals.

There are many different kinds of disabilities, however, there is a general agreement to categorize people with disability into four general categories: visual, auditory, motor (also referred to as “physical”), and cognitive. There are different conditions and much variability within each category, e.g., within visual disabilities, color blindness, low vision, and blindness. There is also a distinction as to when a disability is contracted, e.g., a person who was born blind as opposed to one who lost vision later on in life.

Furthermore, as we age or encounter unique situations (such as multi-tasking), we may have a similar experience to people we think of as disabled. Therefore, disabilities should be thought of as a spectrum of abilities that should be accounted for during the design of all user interfaces and experiences.

Typically, in order to ensure that disabled people can use their digital products and services, companies aim for compliance with accessibility guidelines such as the Web Content Accessibility Guidelines (WCAG 2.0). While this is critical, it is also important to have users with disabilities try to accomplish real tasks on the site in usability testing. There may be gaps in the overall user experience…

Think about the typical doors found in buildings. How many times have you tried to open a door one way and realized they actually open the other, for example, push instead of pull. Technically the door is accessible, but it is usable?


Example of doors that are technically accessible, but not usable
Example of doors that are technically accessible, but not usable (handles give impression you should push but must pull to open). (Large preview)

Even if a site follows the accessibility guidelines and is technically accessible, users may not be able to accomplish their goals on the site.

Lesson Learned

In most ways, usability testing with this segment of the population is no different than testing with anyone else. However, there are several areas you need to pay just a bit more attention to so your sessions run smoothly. The lessons or tips are broken down into general ones that can apply to all participants and specific tips for various disability types such as visual, motor, and cognitive.

General Lessons Learned

1. Ensure a baseline level of accessibility before usability testing

Ensure a baseline level of accessibility before usability testing: Planning usability testing, especially recruiting participants can take time both for the project team and the recruited participants.

Two good examples of basic accessibility issues that should be addressed prior to usability testing are:

  • Missing alternative (alt) text.
    Usability testing can be used to see if the alt text used is appropriate and makes sense to participants, but if all the participants are doing is confirming that the alt text is missing then this is not a good use of their time.
  • Appropriate color contrast.
    All page designs should be reviewed beforehand to make sure all foreground and background colors meet WCAG 2.0 AA color contrast ratios.
2. Focus the recruiting strategy

If you work with an external recruiter ask them if they have experience recruiting people with disabilities; some do. If you are recruiting internally (without an external recruiter), you may need to reach out to organizations that have access to people with disabilities. For example, if you need to recruit participants with visually disabilities in the United States, you should contact a local chapter of the National Federation of the Blind (https://nfb.org/state-and-local-organizations) or a local training center such as the Carroll Center for the Blind in Massachusetts (http://carroll.org/). If you use social media to advertise your study, a good approach is to use the hashtag #a11y (stands for accessibility — there are 11 letters between the “a” and “y”) in your post.

3. Bring their own equipment/assistive technology

Allow and encourage participants to bring their own equipment such as their own laptop, especially if they use assistive technology. This way, you can truly see how people customize and use assistive technology.

4. Have a backup plan for assistive technology

As stated above in #3. It is best if participants can bring their own equipment. However, it is always wise to plan for the worst, for example, if a participant does not bring their equipment or if there is a technical problem such as you can’t connect their equipment to your Wi-Fi network. In the case of visually impaired participants, install assistive technology (AT) such as screen reader software they will be bringing in on a backup PC. For many of the AT software packages, you can get a free trail that should cover you for the usability testing period. This has saved us several times. Even though the configuration was different than what the participants had, we were able to run the session. Participants were quickly able to go into the settings and make some adjustments (e.g., increase the speech rate) and get started with the session.