Rosenverse
SUS: A System Unusable for Twenty Percent of the Population

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

SUS: A System Unusable for Twenty Percent of the Population

Gold
Thursday, December 9, 2021 • Civic Design 2021
Share the love for this talk
SUS: A System Unusable for Twenty Percent of the Population
Speakers: Sam Proulx
Link:

Summary

Did you know that the SUS creates biases in your research, affecting one in five people? That’s right!  People with disabilities, especially those who use assistive technology, are not considered by most of the questions in the SUS. As a designer for the public sector, this could lead to you making design decisions that don’t take into account 20 percent of the visitors to your website.

Key Insights

  • •

    Screen readers allow users to navigate web pages quickly by jumping through well-structured headings and content.

  • •

    Low vision users often rely on system-wide screen magnification and color inversion settings to reduce eye strain.

  • •

    Alternative navigation methods like head mice and voice dictation are selected based on task and user condition, not used exclusively.

  • •

    Traditional lab-based usability research often prioritizes convenience over real assistive technology configurations, leading to inaccurate results.

  • •

    Assistive technology users rarely use default settings; they extensively customize tools over time for optimal use.

  • •

    The System Usability Scale (SUS) includes questions that confuse assistive technology users and do not capture their true experience.

  • •

    Fable’s Accessible Usability Scale adapts SUS questions to better reflect the experience of assistive technology users.

  • •

    Initial data shows screen reader users report significantly lower usability scores (mid-50s) compared to screen magnification users (low 60s).

  • •

    Users’ lowered expectations after years of experience may mask the true severity of usability issues in assistive technology contexts.

  • •

    Measuring accessibility usability accurately enables benchmarking and progress in improving digital products for all users.

Notable Quotes

"I’ve been using screen readers for 30 years and accessibility has always been a passion of mine."

"Screen readers let us jump from one heading to another, replicating the experience of skimming a page visually."

"Low vision users often invert colors and use magnification levels as high as 400 percent to reduce eye strain."

"Alternative navigation users pick different tools based on the task and how they feel at that moment, not just one technology."

"In a usability session, the biggest challenge is not to talk at the same time the screen reader is talking."

"No assistive technology user uses default settings; we spend years customizing everything to work for us."

"The System Usability Scale’s question about needing a technical person to use the system is confusing and inaccurate for screen reader users."

"Most people with my assistive technology might learn to use the system at a different pace than non-disabled users."

"Screen reader users scored significantly lower in usability tests, reflecting the most effort in interpreting content."

"If you’re doing research with people with disabilities, consider using the Accessible Usability Scale to get more accurate insights."

Ask the Rosenbot
Saara Kamppari-Miller
Key Metrics: Comparing Three Letter Acronym Metrics That Include the Word “Key”
2024 • DesignOps Community
Matt Duignan
Atomizing Research: Trend or Trap
2020 • Advancing Research 2020
Gold
Laine Riley Prokay
Carving a Path for Early Career DesignOps Practitioners
2022 • DesignOps Summit 2022
Gold
Greg Petroff
Exit Interview #1: Greg Petroff: From Silicon Valley Executive to Sonoma County Possibilitarian
2025 • Rosenfeld Community
Anne Mamaghani
How Your Organization's Generative Workshops Are Probably Going Wrong and How to Get Them Right
2023 • Advancing Research 2023
Gold
Samuel Proulx
Invisible barriers: Why accessible service design can’t be an afterthought
2024 • Advancing Service Design 2024
Gold
Bianca Jefferson
From Sprints to Systems: Operationalizing Continuous Discovery Through DesignOps
2025 • DesignOps Summit 2025
Gold
Theresa Marwah
How Atlassian is Operationalizing Respect in Research
2020 • Advancing Research Community
JJ Kercher
A Roadmap for Maturing Design in the Enterprise
2018 • Enterprise Experience 2018
Gold
Samuel Martin
Co-Design vs Faux-Design: Navigating the Complexities of Sharing Power in Co-Design
2026 • Rosenfeld Community
Maria Giudice
Becoming a Changemaker by Leading with Design
2023 • Advancing Research 2023
Gold
Max Gadney
Assessing UX jobs for impact in climate
2024 • Climate UX Interest Group
Marisa Bernstein
It Takes GRIT: Lessons from the Small, but Mighty World of Civic Usability Testing
2021 • Civic Design 2021
Gold
Sean McKay
Whole Product Thinking: Expanding beyond problem and solution space thinking
2024 • Rosenfeld Community
Stephen Anderson
Puzzled? How to Coordinate Humans for Complex Challenges
2021 • Enterprise Community
Francesca Barrientos, PhD
You Need Your Own Definition of Design Maturity
2022 • Design at Scale 2022
Gold

More Videos

Shipra Kayan

"We moved from Evernote to spreadsheets, to a giant affinity board in Mirror, then to UserVoice, and finally to idiomatic software with AI."

Shipra Kayan

How we Built a VoC (Voice of the Customer) Practice at Upwork from the Ground Up

September 30, 2021

Peter Morville

"Every few years consultants come in and tidy up our mess, but as soon as they leave, we mess it all up again."

Peter Morville

The Architecture of Understanding

May 13, 2015

Renee Reid

"I was able to connect the dots a lot for the UX team because I had those relationships from customer success."

Renee Reid

Becoming a ResearchH.E.R (Highly Enterprise Ready)

June 3, 2019

Dane DeSutter

"Physical therapists see their hands as a golden resource and tend to resist technology that might replace that."

Dane DeSutter

Keeping the Body in Mind: What Gestures and Embodied Actions Tell You That Users May Not

March 26, 2024

Sam Proulx

"Avoid strictly timed interactions because they stress everyone, especially people using screen readers or with cognitive disabilities."

Sam Proulx

Online Shopping: Designing an Accessible Experience

October 3, 2023

Andrew Custage

"While technical errors matter, broader issues like content relevance and navigation create more mission failures."

Andrew Custage Michael Mallett

The Digital Journey: Research on Consumer Frustration and Loyalty

March 29, 2023

Shelby Switzer

"Facilitation is part setting the stage and part performing on it."

Shelby Switzer

Making Space for Community Knowledge-sharing in a Distributed World

December 10, 2021

Llewyn Paine

"If we are haphazard about AI adoption and trust marketers over rigorous testing, it looks like all our hand wringing about research accuracy was just gatekeeping."

Llewyn Paine

Coexisting with AI: A practical guide for researchers to navigate tools, ethics, and integration

March 11, 2025

Robin Beers

"Simon Roberts calls this the UX ification of research, which leads to a more transactional focus on informing technology requirements."

Robin Beers

Navigating organizational systems: Rethinking researcher’s role in driving change

March 13, 2025