Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Does Designing and Researching Data Products Powered by ML/AI and Analytics Call for New UX Methods?
Friday, February 18, 2022 • QuantQual Interest Group (Rosenfeld Community)
Share the love for this talk
Does Designing and Researching Data Products Powered by ML/AI and Analytics Call for New UX Methods?
Speakers: Brian T. O’Neill , Maria Cipollone , Luis Colin , Manuel Dahm and Mike Oren
Link:

Summary

Join us for a different type of Quant vs. Qual discussion: instead of discussing how data science and quantitative research methods can power UX research and design, we’re going to talk about designing enterprise data products and tools that put ML and analytics into the hands of users. Does this call for new, different, or modified approaches to UX research and design? Or do these technologies have nothing to do with how we approach design for data products? The session’s host will be Brian T. O’Neill who is also the host of the Experiencing Data podcast and founder of Designing for Analytics, an independent consultancy that helps data products leaders use design-driven innovation to deliver better ML and analytics user experiences. In this session, we’ll be sharing some rapid [slides-optional] anecdotes and stories from the attendees and then open up the conversation to everyone. We hope to get perspectives from both enterprise data teams doing “internal” data analytics or ML/AI solutions development as well as software/tech companies as well who may offer data-related platform tools, intelligence/SAAS products, BI/decision support solutions, etc. Slots are open to both experienced UX practitioners as well as data science / analytics / technical participants who may have participated in design or UX work with colleagues. Please share! If folks are too quiet in the session, you may be subject to a drum or tambourine solo from Brian. Nobody has all of this “figured out yet” and experiments and trials are welcome.

Key Insights

  • UX researchers working with ML and data science teams often lack domain expertise, requiring strong facilitation and interpretive interview skills.

  • Machine learning product users are usually small specialized expert groups, making statistical rigor and sampling difficult.

  • Trust and interpretability in ML models can be more important than peak accuracy for business adoption.

  • Collaborative shared spaces or cross-functional teams focused on common goals bridge gaps between design, engineering, and data science.

  • Designers help translate data science outputs into actionable, contextual decision support for end users.

  • Prototyping ML products requires believable, realistic data and testing boundaries like false positives early to build trust.

  • Decision culture (focusing on which decisions to support) is a more useful framing than data culture in enterprise AI/ML products.

  • The ecosystem around ML includes not only users and business stakeholders but also data labelers who impact outcomes.

  • The integration of business rules with ML models enhances contextual relevance and facilitates user trust.

  • No-code and rapid data science tools are emerging to help speed experimentation but do not fully replace model development demands.

Notable Quotes

"When you work with machine learning engineers doing advanced techniques, you’re really out of the realm of your knowledge."

"Most data science and analytics teams do not have designers or user experience people unless they are software native companies."

"If nobody uses this because they don’t trust it, it doesn’t matter. You just rehearsed without a concert."

"We need to think more about decision culture instead of data culture — what decisions are we trying to make?"

"Sometimes, the answer is ignore machine learning here. It is not the right tool for everything."

"Designers partnering with data scientists leads to smarter, more adaptable interfaces that actually get used."

"We’re talking to really small groups — sometimes 8 to 10 ML engineers on a particular domain — so sampling and rigor are tough."

"Creating that shared space where design, engineering, and data science work together is key to success."

"The human algorithms that people use today should be understood and incorporated into ML models where possible."

"You have to test the boundaries of false positives, false negatives, and surprising positives to see if people trust your model."

Ask the Rosenbot
Ned Gartside
Navigating accessibility and climate (Videoconference)
2024 • Climate UX Interest Group (Rosenfeld Community)
Ana Ferreira
Designing Distributed: Leading Doist’s Fully Remote Design Team in Six Countries
2024 • DesignOps Summit 2020
Gold
Peter Merholz
The Mysterious Case of the Missing UX Career Path (Videoconference)
2022 • DesignOps Community
Uday Gajendar
Leading through the long tail of trauma (Videoconference)
2022 • Advancing Research Community
Kevin M. Hoffman
Theme 2: Enterprise Team Journey
2019 • Enterprise Experience 2019
Gold
Rich Mironov
How Can Product Managers and UXers Help Each Other (and Why are Product Folks so Annoying Sometimes)?
2022 • Design in Product 2022
Gold
Mujtaba Hameed
Frameworks for Excellence: Using Visual Thinking and Communication to Elevate Your Research
2024 • Advancing Research 2024
Gold
Miles Orkin
Creativity and Culture
2018 • DesignOps Summit 2018
Gold
Meredith Black
Scaling Design Culture
2017 • DesignOps Summit 2017
Gold
Devon Powers
Imagining Better Futures
2022 • Advancing Research 2022
Gold
Bria Alexander
Welcome
2024 • Enterprise Experience 2020
Gold
Christian Crumlish
AMA with Christian Crumlish, author of Product Management for UX People (Videoconference)
2022 • Enterprise Community
Andrew Michael
Building a Product Insights Team
2022 • Advancing Research 2022
Gold
Anat Fintzi
Delivering at Scale: Making Traction with Resistant Partners
2022 • Design at Scale 2022
Gold
Louis Rosenfeld
Founder’s Welcome
2022 • Design in Product 2022
Gold
Rusha Sopariwala
Remote, Together: Craft and Collaboration Across Disciplines, Borders, Time Zones, and a Design Org of 170+
2022 • Design at Scale 2022
Gold

More Videos

Adam Cutler

"I wish I could tell my 22-year-old self what to stop doing and what to embrace to be a better designer."

Adam Cutler Karen Pascoe Ian Swinson Susan Worthman

Discussion

June 8, 2016

Peter Merholz

"Knowing your subject matter as a UX leader is just as important as knowing your craft."

Peter Merholz

The Trials and Tribulations of Directors of UX (Videoconference)

July 13, 2023

Lisa Welchman

"Digital is a system, not a project. It’s there all the time and you have to keep iterating on it."

Lisa Welchman

Cleaning Up Our Mess: Digital Governance for Designers

June 14, 2018

Vincent Brathwaite

"Sustainable practices are not just a luxury; they are a necessity for our survival."

Vincent Brathwaite

Opener: Past, Present, and Future—Closing the Racial Divide in Design Teams

October 22, 2020

Brenna Fallon

"Design your processes around learning, have blameless post mortems and celebrate failures especially."

Brenna Fallon

Learning Over Outcomes

October 24, 2019

Tricia Wang

"Working from home during the pandemic is hard because it’s fun only when you can actually leave your home."

Tricia Wang

Spatial Collapse: Designing for Emergent Culture

January 8, 2024

Edgar Anzaldua Moreno

"We aimed for research that is actionable, not just insightful or pretty to look at."

Edgar Anzaldua Moreno

Using Research to Determine Unique Value Proposition

March 11, 2021

"The toy shouldn’t be the goal of play, but a tool or a process that unlocks the unlimited possibilities set forth before a child."

Designing Systems at Scale

November 7, 2018

Erin Weigel

"You’re never testing an idea purely; you’re always testing the implementation of that idea."

Erin Weigel

Get Your Whole Team Testing to Design for Impact

July 24, 2024