Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Does Designing and Researching Data Products Powered by ML/AI and Analytics Call for New UX Methods?
Friday, February 18, 2022 • QuantQual Interest Group (Rosenfeld Community)
Share the love for this talk
Does Designing and Researching Data Products Powered by ML/AI and Analytics Call for New UX Methods?
Speakers: Brian T. O’Neill , Maria Cipollone , Luis Colin , Manuel Dahm and Mike Oren
Link:

Summary

Join us for a different type of Quant vs. Qual discussion: instead of discussing how data science and quantitative research methods can power UX research and design, we’re going to talk about designing enterprise data products and tools that put ML and analytics into the hands of users. Does this call for new, different, or modified approaches to UX research and design? Or do these technologies have nothing to do with how we approach design for data products? The session’s host will be Brian T. O’Neill who is also the host of the Experiencing Data podcast and founder of Designing for Analytics, an independent consultancy that helps data products leaders use design-driven innovation to deliver better ML and analytics user experiences. In this session, we’ll be sharing some rapid [slides-optional] anecdotes and stories from the attendees and then open up the conversation to everyone. We hope to get perspectives from both enterprise data teams doing “internal” data analytics or ML/AI solutions development as well as software/tech companies as well who may offer data-related platform tools, intelligence/SAAS products, BI/decision support solutions, etc. Slots are open to both experienced UX practitioners as well as data science / analytics / technical participants who may have participated in design or UX work with colleagues. Please share! If folks are too quiet in the session, you may be subject to a drum or tambourine solo from Brian. Nobody has all of this “figured out yet” and experiments and trials are welcome.

Key Insights

  • UX researchers working with ML and data science teams often lack domain expertise, requiring strong facilitation and interpretive interview skills.

  • Machine learning product users are usually small specialized expert groups, making statistical rigor and sampling difficult.

  • Trust and interpretability in ML models can be more important than peak accuracy for business adoption.

  • Collaborative shared spaces or cross-functional teams focused on common goals bridge gaps between design, engineering, and data science.

  • Designers help translate data science outputs into actionable, contextual decision support for end users.

  • Prototyping ML products requires believable, realistic data and testing boundaries like false positives early to build trust.

  • Decision culture (focusing on which decisions to support) is a more useful framing than data culture in enterprise AI/ML products.

  • The ecosystem around ML includes not only users and business stakeholders but also data labelers who impact outcomes.

  • The integration of business rules with ML models enhances contextual relevance and facilitates user trust.

  • No-code and rapid data science tools are emerging to help speed experimentation but do not fully replace model development demands.

Notable Quotes

"When you work with machine learning engineers doing advanced techniques, you’re really out of the realm of your knowledge."

"Most data science and analytics teams do not have designers or user experience people unless they are software native companies."

"If nobody uses this because they don’t trust it, it doesn’t matter. You just rehearsed without a concert."

"We need to think more about decision culture instead of data culture — what decisions are we trying to make?"

"Sometimes, the answer is ignore machine learning here. It is not the right tool for everything."

"Designers partnering with data scientists leads to smarter, more adaptable interfaces that actually get used."

"We’re talking to really small groups — sometimes 8 to 10 ML engineers on a particular domain — so sampling and rigor are tough."

"Creating that shared space where design, engineering, and data science work together is key to success."

"The human algorithms that people use today should be understood and incorporated into ML models where possible."

"You have to test the boundaries of false positives, false negatives, and surprising positives to see if people trust your model."

Milan Guenther
A Shared Language for Co-Creating Ambitious Endeavours
2023 • Enterprise UX 2023
Gold
Dave Hoffer
UX Job Search AMA with Joanne Weaver and Dave Hoffer
2025 • Rosenfeld Community
Jorge Arango
[Demo] How to re-categorize content at scale using LLMs
2024 • Designing with AI 2024
Gold
Jon Fukuda
Theme 3 Intro
2024 • DesignOps 2024
Gold
Lada Gorlenko
Theme 1: Discussion
2024 • Enterprise Experience 2020
Gold
Jemma Ahmed
Theme Three Intro
2023 • Advancing Research 2023
Gold
Ariel Kennan
Theme 2 Intro
2021 • Civic Design 2021
Gold
Bria Alexander
Opening Remarks
2021 • DesignOps Summit 2021
Gold
Louis Rosenfeld
Coffee with Lou (Videoconference)
2024 • Rosenfeld Community
Ari Zelmanow
Dark Side of Democratization (Videoconference)
2023 • Advancing Research Community
Gina Mendolia
Therapists, Coaches, and Grandmas: Techniques for Service Design in Complex Systems
2024 • Advancing Service Design 2024
Gold
Brianna Sylver
Lead With Purpose
2020 • Advancing Research 2020
Gold
Ryan Rumsey
Business Influence Without Losing Your Soul (Videoconference)
2021 • Enterprise Community
Chris Chapo
Data Science and Design: A Tale of Two Tribes
2015 • Enterprise UX 2015
Gold
Anat Fintzi
Delivering at Scale: Making Traction with Resistant Partners
2022 • Design at Scale 2022
Gold
Dane DeSutter
Beyond the Console: The rise of the Gamer Experience and how gaming will impact UX Research across industries (Videoconference)
2024 • QuantQual Interest Group (Rosenfeld Community)

More Videos

Monty Hammontree

"Our mission is to empower everybody to learn, not just have a few people learning on behalf of the rest."

Monty Hammontree

The Future of UX Research (Videoconference)

December 3, 2020

Liz Ebengo

"You have to engage every layer of the system—from local communities to NGOs and government power structures—to create change."

Liz Ebengo

The Burden on Children: The Cost of Insufficient Post-Conflict Services and Pathways Forward

December 4, 2024

Billy Carlson

"I tell people to find inspiration from existing designs and bring those into your wireframes to look like real products."

Billy Carlson

Tips to Utilize Wireframes to Tell an Effective Product Story

June 6, 2023

Saara Kamppari-Miller

"Design operations is designing how we design, and inclusive design is design ops we need to be intentional about."

Saara Kamppari-Miller

Inclusive Design is DesignOps

September 29, 2021

Ian Swinson

"Leadership does not mean standing in front of a room. It means mentoring, influencing, and helping others grow."

Ian Swinson

Designing and Driving UX Careers

June 8, 2016

Jonathon Colman

"Working across multiple teams all the time is normal for content designers, but that doesn’t mean it’s the best or the only way."

Jonathon Colman

How to Maximize the Impact of Content Design

January 8, 2024

Maria Skaaden

"Opening the feedback board to the whole team made developers feel responsible and inspired to solve user needs."

Maria Skaaden

Continuous Design: One eye on the horizon and the other on the next wave

November 8, 2018

Bryce Benton

"Markdown is a very concise and useful format, and it's used all over the place on GitHub."

Bryce Benton

[Demo] AI-powered UX enhancement: Aligning GitHub documentation with USWDS at Austin Public Library

June 4, 2024

Marc Fonteijn

"Men in this report consistently report higher income than women, but don’t draw early conclusions — seniority levels and positions matter."

Marc Fonteijn

First Insights from the 2025 Service Design Salary(+) Report

December 4, 2024