Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

SUS: A System Unusable for Twenty Percent of the Population

Gold
Thursday, December 9, 2021 • Civic Design 2021
Share the love for this talk
SUS: A System Unusable for Twenty Percent of the Population
Speakers: Sam Proulx
Link:

Summary

Did you know that the SUS creates biases in your research, affecting one in five people? That’s right!  People with disabilities, especially those who use assistive technology, are not considered by most of the questions in the SUS. As a designer for the public sector, this could lead to you making design decisions that don’t take into account 20 percent of the visitors to your website.

Key Insights

  • Screen readers allow users to navigate web pages quickly by jumping through well-structured headings and content.

  • Low vision users often rely on system-wide screen magnification and color inversion settings to reduce eye strain.

  • Alternative navigation methods like head mice and voice dictation are selected based on task and user condition, not used exclusively.

  • Traditional lab-based usability research often prioritizes convenience over real assistive technology configurations, leading to inaccurate results.

  • Assistive technology users rarely use default settings; they extensively customize tools over time for optimal use.

  • The System Usability Scale (SUS) includes questions that confuse assistive technology users and do not capture their true experience.

  • Fable’s Accessible Usability Scale adapts SUS questions to better reflect the experience of assistive technology users.

  • Initial data shows screen reader users report significantly lower usability scores (mid-50s) compared to screen magnification users (low 60s).

  • Users’ lowered expectations after years of experience may mask the true severity of usability issues in assistive technology contexts.

  • Measuring accessibility usability accurately enables benchmarking and progress in improving digital products for all users.

Notable Quotes

"I’ve been using screen readers for 30 years and accessibility has always been a passion of mine."

"Screen readers let us jump from one heading to another, replicating the experience of skimming a page visually."

"Low vision users often invert colors and use magnification levels as high as 400 percent to reduce eye strain."

"Alternative navigation users pick different tools based on the task and how they feel at that moment, not just one technology."

"In a usability session, the biggest challenge is not to talk at the same time the screen reader is talking."

"No assistive technology user uses default settings; we spend years customizing everything to work for us."

"The System Usability Scale’s question about needing a technical person to use the system is confusing and inaccurate for screen reader users."

"Most people with my assistive technology might learn to use the system at a different pace than non-disabled users."

"Screen reader users scored significantly lower in usability tests, reflecting the most effort in interpreting content."

"If you’re doing research with people with disabilities, consider using the Accessible Usability Scale to get more accurate insights."

Ask the Rosenbot
Jake Burghardt
Stop wasting research: Unlock more value from research insights
2025 • Rosenfeld Community
Lada Gorlenko
Theme 3: Introduction
2021 • Design at Scale 2021
Gold
Maggie Dieringer
Creating Consistency Through Constant Change
2024 • DesignOps Summit 2020
Gold
Cheryl Platz
Merging Improv with Design
2019 • Enterprise Community
Dalia El-Shimy
So You've Got a Seat at the Table. Now What?
2020 • Advancing Research 2020
Gold
Mandy Drew
What Role(s) Can Research Play in Responsible Design?
2021 • Advancing Research 2021
Gold
Bria Alexander
Opening Remarks
2021 • Design at Scale 2021
Gold
Sheri Byrne-Haber
Accessibility at Scale
2021 • Design at Scale 2021
Gold
Cennydd Bowles
Responsible Design in Reality
2021 • Design at Scale 2021
Gold
Davis Neable
How to Drive a Design Project When you Don’t Have a Design Team
2021 • Design at Scale 2021
Gold
Nathan Shedroff
How Will Design be Taught When the Schools Shut Down?
2026 • Rosenfeld Community
Stephen Anderson
Puzzled? How to Coordinate Humans for Complex Challenges
2021 • Enterprise Community
Sam Proulx
Designing For Screen Readers: Understanding the Mental Models and Techniques of Real Users
2021 • DesignOps Summit 2021
Gold
Scott Stephens
The Next Generation in DesignOps Toolsets
2022 • DesignOps Community
Anna Avrekh
Diversity In and For Design: Building Conscious Diversity in Design and Research
2021 • Design at Scale 2021
Gold
John Calhoun
Have we Reached Our Peak? Spotting the Next Mountain For DesignOps to Climb
2021 • DesignOps Summit 2021
Gold

More Videos

Joanna Vodopivec

"Collaboration with another researcher was seamless because we speak the same methodology language and aligned on goals."

Joanna Vodopivec Prabhas Pokharel

One Research Team for All - Influence Without Authority

March 9, 2022

Louis Rosenfeld

"Onboarding often felt like trial by fire—go to the meeting and hope you understand something."

Louis Rosenfeld Lashanda Hodge Senongo Akpem Chris Hodowanec

Becoming a Civic Designer: Making the Move from Private to Public Sector

November 17, 2022

Bria Alexander

"Please put your questions for a specific speaker in the associated thread for the talk."

Bria Alexander

Day 3 Welcome

September 25, 2024

Uday Gajendar

"Lauren Cantor is a heavy hitter gathering all the really good resources and putting them together."

Uday Gajendar Louis Rosenfeld

Day 2 Welcome

June 5, 2024

Sam Proulx

"I’ve nearly bought products in my sleep because I memorized the key presses from consistent checkout flows."

Sam Proulx

Online Shopping: Designing an Accessible Experience

June 7, 2023

Russ Unger

"Buddy systems create leadership opportunities and a support network without changing org charts."

Russ Unger

Onboarding: The Ecosystem, not the Afterthought

November 7, 2017

Catherine Dubut

"No one really knows what to expect anyway, so failure goes largely unnoticed. It's better to be transparent about the mess."

Catherine Dubut

Bridging Physical and Digital Spaces: Approaches to Retail Service Design

March 18, 2021

Josh Clark

"We think of AI not just as a maker of stuff but as an enabler of experiences."

Josh Clark Veronika Kindred

Sentient Design: New Postures for AI-Mediated Experiences (2nd of 3 seminars)

January 29, 2025

Dantley Davis

"My dad told me very early on in my career that I had to work two times harder than anyone else to get the same things."

Dantley Davis

Leadership & Diversity—A Fireside Chat with Dantley Davis

September 17, 2020