Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Does Designing and Researching Data Products Powered by ML/AI and Analytics Call for New UX Methods?
Friday, February 18, 2022 • QuantQual Interest Group (Rosenfeld Community)
Share the love for this talk
Does Designing and Researching Data Products Powered by ML/AI and Analytics Call for New UX Methods?
Speakers: Brian T. O’Neill , Maria Cipollone , Luis Colin , Manuel Dahm and Mike Oren
Link:

Summary

Join us for a different type of Quant vs. Qual discussion: instead of discussing how data science and quantitative research methods can power UX research and design, we’re going to talk about designing enterprise data products and tools that put ML and analytics into the hands of users. Does this call for new, different, or modified approaches to UX research and design? Or do these technologies have nothing to do with how we approach design for data products? The session’s host will be Brian T. O’Neill who is also the host of the Experiencing Data podcast and founder of Designing for Analytics, an independent consultancy that helps data products leaders use design-driven innovation to deliver better ML and analytics user experiences. In this session, we’ll be sharing some rapid [slides-optional] anecdotes and stories from the attendees and then open up the conversation to everyone. We hope to get perspectives from both enterprise data teams doing “internal” data analytics or ML/AI solutions development as well as software/tech companies as well who may offer data-related platform tools, intelligence/SAAS products, BI/decision support solutions, etc. Slots are open to both experienced UX practitioners as well as data science / analytics / technical participants who may have participated in design or UX work with colleagues. Please share! If folks are too quiet in the session, you may be subject to a drum or tambourine solo from Brian. Nobody has all of this “figured out yet” and experiments and trials are welcome.

Key Insights

  • UX researchers working with ML and data science teams often lack domain expertise, requiring strong facilitation and interpretive interview skills.

  • Machine learning product users are usually small specialized expert groups, making statistical rigor and sampling difficult.

  • Trust and interpretability in ML models can be more important than peak accuracy for business adoption.

  • Collaborative shared spaces or cross-functional teams focused on common goals bridge gaps between design, engineering, and data science.

  • Designers help translate data science outputs into actionable, contextual decision support for end users.

  • Prototyping ML products requires believable, realistic data and testing boundaries like false positives early to build trust.

  • Decision culture (focusing on which decisions to support) is a more useful framing than data culture in enterprise AI/ML products.

  • The ecosystem around ML includes not only users and business stakeholders but also data labelers who impact outcomes.

  • The integration of business rules with ML models enhances contextual relevance and facilitates user trust.

  • No-code and rapid data science tools are emerging to help speed experimentation but do not fully replace model development demands.

Notable Quotes

"When you work with machine learning engineers doing advanced techniques, you’re really out of the realm of your knowledge."

"Most data science and analytics teams do not have designers or user experience people unless they are software native companies."

"If nobody uses this because they don’t trust it, it doesn’t matter. You just rehearsed without a concert."

"We need to think more about decision culture instead of data culture — what decisions are we trying to make?"

"Sometimes, the answer is ignore machine learning here. It is not the right tool for everything."

"Designers partnering with data scientists leads to smarter, more adaptable interfaces that actually get used."

"We’re talking to really small groups — sometimes 8 to 10 ML engineers on a particular domain — so sampling and rigor are tough."

"Creating that shared space where design, engineering, and data science work together is key to success."

"The human algorithms that people use today should be understood and incorporated into ML models where possible."

"You have to test the boundaries of false positives, false negatives, and surprising positives to see if people trust your model."

Tamara Hale
War Stories LIVE! Tamara Hale
2020 • Advancing Research 2020
Gold
Saara Kamppari-Miller
Theme Three Intro
2023 • DesignOps Summit 2023
Gold
Aurobinda Pradhan
Introduction to Collaborative DesignOps using Cubyts
2022 • DesignOps Summit 2022
Gold
Sean Dolan
A Practical Look at Creating More Usable Enterprise Customer Journeys (Videoconference)
2019 • Enterprise Community
Leisa Reichelt
Opening Keynote: Operating in Context
2018 • DesignOps Summit 2018
Gold
JD Buckley
COMMUNICATE: Discussion
2018 • Enterprise Experience 2018
Gold
Victor Udoewa
Theme One Intro
2023 • Advancing Research 2023
Gold
Candace Myers
Standardizing Design at Scale
2022 • DesignOps Summit 2022
Gold
Mark Templeton
Creating a Legacy: the ultimate experience
2017 • Enterprise Experience 2017
Gold
Frances Yllana
DesignOps–Leading the Path to Parity (Videoconference)
2023 • DesignOps Community
George Abraham
Design Systems To-Go: Introducing a Starter Design System, and Indigo.Design Overview (Part 1)
2021 • DesignOps Summit 2021
Gold
Luz Bratcher
This Is a Talk for Tired People
2022 • Design at Scale 2022
Gold
Peter Merholz
Design at Scale is People!
2021 • Design at Scale 2021
Gold
Aras Bilgen
Research Democratization: A Debate
2023 • Advancing Research 2023
Gold
Noel Lamb
Cultivating Business Partnerships to Grow Research Ops (Videoconference)
2022 • Advancing Research Community
Catherine Courage
The Enterprise UX Journey: Lessons From the Voyage & The Opportunity Ahead
2015 • Enterprise UX 2015
Gold

More Videos

Daniela Magaña Flores

"Users typically rate a product better than non-users because of familiarity and built-up trust."

Daniela Magaña Flores Ariane Rahn Jeff Ephraim Bander

Ahead of Competition: Learn What UX Benchmarking Can Do for Your Business Today

March 10, 2022

Savina Hawkins

"When humans blindly prompt a large language model, they’re not receiving wisdom from an all-knowing oracle but probabilistic outcomes based on patterns."

Savina Hawkins

Harnessing AI in UXR: Practical Strategies for Positive Impact

March 26, 2024

Alex Hurworth

"Each species lost is a thread untethered from the web of life that supports us all."

Alex Hurworth Bonnie John Fahd Arshad Antoine Marin

Designing a Contact Tracing App for Universal Access

October 23, 2020

Matt Duignan

"Researchers often struggle to understand why past design decisions were made because knowledge is siloed or buried."

Matt Duignan

HITS, Microsoft's internal human insight system: From research library to living body of knowledge (Videoconference)

July 16, 2019

Craig Villamor

"Designers need to speak truth to power and engage their creative abilities to materially change how and what is produced - Diego Rodriguez."

Craig Villamor

Design Systems for Ethical Design (Videoconference)

January 26, 2023

Ovetta Sampson

"Think of UX as risk mitigation, not a zero-sum game, and share the risk with your stakeholders."

Ovetta Sampson

Turning UX Passion into Real Product Influence

June 7, 2023

Rima Campbell

"If you’re learning something at the end of Sprint six, it’s really costly to go back and make changes in Sprint one or two."

Rima Campbell Amrit S Bhachu

Increase Productivity and Drive Business Impact

September 24, 2024

Amy Gawronski Zuccaro

"Set up clear boundaries early on with your head of design on what lives in your world and what lives in theirs."

Amy Gawronski Zuccaro

Advice for DesignOps Employee #1

September 29, 2021

Victor Udoewa

"Will you learn to let go? Better yet, will you let go?"

Victor Udoewa

Beyond Methods and Diversity: The Roots of Inclusion

March 26, 2024