Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

SUS: A System Unusable for Twenty Percent of the Population
Gold
Wednesday, June 9, 2021 • Design at Scale 2021
Share the love for this talk
SUS: A System Unusable for Twenty Percent of the Population
Speakers: Sam Proulx
Link:

Summary

Did you know that the SUS creates biases in your research, affecting one in five people? That’s right! People with disabilities, especially those who use assistive technology, are not considered by most of the questions in the SUS. When the SUS was invented, the author encouraged people to change it to suit different needs. In this talk, Samuel Proulx and Abid Virani from Fable will discuss how Fable adapted the SUS to work for assistive technology users. Drawing from over five thousand hours of research and testing involving assistive technology users, we created the Accessible Usability Scale (AUS). This presentation will include trends in AUS responses since it was released in December of 2020.

Key Insights

  • Assistive technology users fall broadly into three groups: screen reader users, screen magnification users, and alternative navigation users.

  • Screen reader users often face the greatest usability challenges due to lack of backup navigation options.

  • Screen magnification tools sometimes incorporate screen reader features to reduce eye strain and aid users with partial vision loss.

  • Alternative navigation users leverage multiple input methods, including voice commands, head mice, and coordinate grids, to overcome interaction barriers.

  • Traditional usability labs are poorly suited for testing with assistive technology users because they use unfamiliar setups that aren't customized.

  • Remote usability testing better captures authentic user experiences by allowing assistive technology users to test in their optimized environments.

  • The original System Usability Scale (SUS) includes language and assumptions that are often unclear or irrelevant to assistive technology users.

  • Fable developed an Accessible Usability Scale (AUS), modifying SUS questions to focus on specific features and removing technical assumptions for better relevance.

  • Experience level with assistive technology and timing of disability onset deeply influence users’ mental models and usability perceptions.

  • Usability should be assessed on both accessibility and efficiency — a feature that takes much longer for assistive technology users to complete may be considered inaccessible.

Notable Quotes

"Assistive technology is a group of technologies that people with disabilities use to access computers, grouped mainly into screen readers, magnification, and alternative navigation."

"The primary customization for screen reader users is speed — expert users can listen to synthetic voices at over 300 words per minute and fully understand."

"Magnification tools may include screen reader features because users can get eye strain and prefer to have content read aloud from a specific spot."

"If a screen reader can’t activate or describe a button, the user has no backup other than getting someone with sight involved."

"Traditional usability labs force assistive technology users to use unfamiliar setups, which can drastically impact their performance."

"Remote testing helps users be in comfortable, accessible environments with fully customized assistive technology setups."

"I am a technical person and I can't use this system — sometimes you don’t need a technical person, you just need someone who can see to help with barriers."

"Accessibility is a spectrum, not a checkbox — some features might require assistance but the rest of the product can be fully accessible."

"We’ve ruled out self-reporting skill levels because novice users often overestimate their expertise and near-experts underestimate theirs."

"It’s not just user skill with the system, but also the broader ways people use technology that determines their experience and accessibility needs."

Ask the Rosenbot
Ben Davies
Expert Panel: The Principles of Research Repository Design
2022 • Advancing Research 2022
Gold
Anna Avrekh
User Research, Design, and Product - A Love Story
2021 • Advancing Research 2021
Gold
Etienne Fang
The Power of Care: From Human-Centered Research to Humanity-Centered Leadership
2021 • Advancing Research 2021
Gold
Matt Stone
Scaling Empathy, A Case Study in Change Management
2021 • Design at Scale 2021
Gold
Louis Rosenfeld
Welcome / Housekeeping
2023 • Enterprise UX 2023
Gold
Erin Weigel
Get Your Whole Team Testing to Design for Impact
2024 • Rosenfeld Community
Dean Broadley
Not Black Enough to be White
2024 • DesignOps Summit 2020
Gold
Kate Stern
Scaling Learning for the Future
2022 • DesignOps Summit 2022
Gold
Matteo Gratton
Can Data and Ethics Live Together?
2021 • DesignOps Summit 2021
Gold
Michelle Morrison
Practice What You Preach
2024 • DesignOps Summit 2020
Gold
Brigette Metzler
Scaling ResearchOps: Helping Researchers do Their Best Work
2020 • Advancing Research 2020
Gold
Amy Gawronski Zuccaro
Advice for DesignOps Employee #1
2021 • DesignOps Summit 2021
Gold
Marina Martin
Lives on the Line: The Stakes of UX at the Scale of Government
2018 • Enterprise Experience 2018
Gold
Saara Kamppari-Miller
Key Metrics: Comparing Three Letter Acronym Metrics That Include the Word “Key”
2024 • DesignOps Community
Randolph Duke II
War Stories LIVE! Randy Duke II
2020 • Advancing Research 2020
Gold
Sofia Quintero
The Product Philosophy Behind EnjoyHQ
2021 • Advancing Research 2021
Gold

More Videos

Jemma Ahmed

"You need well-defined frameworks and guardrails for experimentation success to prevent chaos across teams."

Jemma Ahmed Steve Carrod Chris Geison Dr. Shadi Janansefat Christopher Nash

Democratization: Working with it, not against it [Advancing Research Community Workshop Series]

July 24, 2024

Nina Jurcic

"The work of the design system is constant change management, raising awareness, and fostering common language."

Nina Jurcic

The Design System Rollercoaster: From Enabler and Bottleneck to Catalyst for Change

October 3, 2023

Nathan Curtis

"Design principles are guiding tools for conversation rather than strict rules with objective scores."

Nathan Curtis Nalini P. Kotamraju Jack Moffett Dawn Ressel

Discussion

June 9, 2016

Saara Kamppari-Miller

"The word ‘key’ is an extra super duper important differentiator — among all priorities, these are the keys."

Saara Kamppari-Miller Nicole Bergstrom Shashi Jain

Key Metrics: Comparing Three Letter Acronym Metrics That Include the Word “Key”

November 13, 2024

Malini Rao

"Don’t forget the critical change agent persona — adoption and success was determined by the admin or practitioner users."

Malini Rao

Lessons Learned from a 4-year Product Re-platforming Journey

June 9, 2021

Mackenzie Cockram

"85% of people were leaving after just looking at one page on the old site—that was a huge problem."

Mackenzie Cockram Sara Branco Cunha Ian Franklin

Integrating Qualitative and Quantitative Research from Discovery to Live

December 16, 2022

Bria Alexander

"If you signed up for a cohort, please make sure you’re in your private cohort channel to meet and discuss with your group."

Bria Alexander

Opening Remarks

June 9, 2021

Jackie Ho

"Growth boards are a lean governance tool borrowed from the mindset of venture capital."

Jackie Ho

Lead Effectively While Preserving Team Autonomy with Growth Boards

January 8, 2024

Dan Hill

"When they become important is when they recede into the background and become mundane."

Dan Hill

Designing for the infrastructures of everyday life

June 4, 2024