Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Measuring What Matters
Gold
Wednesday, October 23, 2019 • DesignOps Summit 2019
Share the love for this talk
Measuring What Matters
Speakers: Kristin Wisnewski
Link:

Summary

Research and analytics are the eyes and ears of DesignOps. Validating whether strategies are effective or ineffective allows course correction and adaptation. Identifying problems that require action is essential for facilitating collaboration and ensuring that team members are engaged. Prioritizing user needs is the heart of user experience and Agile development. None of this can be done well without an effective research program. We’ll share how we scale research efforts to collect feedback from thousands of users per month and track the impact of dozens of projects involving 120+ designers.

Key Insights

  • IBM’s CIO IT department is only 3% of the company but manages IT for 12,000 IBMers; the CIO design team is just 1% of that group.

  • Over half of IBM’s 385,000 employees have been hired within five years, introducing ongoing challenges in onboarding and experience design.

  • 30% of IBMers work remotely or offsite, necessitating resilient, intuitive digital tools without onsite support.

  • The team uses an IT pyramid of pain inspired by Maslow’s hierarchy to ensure foundational tech and network reliability before focusing on tasks.

  • Continuous measurement includes tracking NPS, goal completion, ease of use, and user comments linked directly to employee IDs for rich insights.

  • IBM shifted from infrequent large surveys to daily embedded Voice of the Employee forms capturing immediate feedback on 4,000 submissions/month.

  • The design ops team integrates research, UX, content, and coaching to foster collaboration and prevent silos.

  • A predictive AI model helps identify when employees should upgrade their hardware before critical failures occur, leading IBM to offer on-demand new machines.

  • Transparency in metrics drives improvement and accountability, with NPS trending to be incorporated into employee evaluations starting next year.

  • Kristen emphasizes embracing chaos and iteration in design ops rather than forcing rigid processes, highlighting the team's evolution from phase one simplicity to complex maturity.

Notable Quotes

"This small but mighty CIO design team is literally 1% of the 3% IT population."

"Over half of IBMers have been here less than five years, adding complexity to designing their experience."

"The state of IT is a daily reflection of what IBM thinks and feels about its employees."

"You can’t just focus on the task level, you have to consider the entire ecosystem and its layers."

"People take out a pen when the CIO says I have a design user experience and research function reporting directly to me."

"We’re here to make work better, and the coaching tribe is here to make teams better."

"We moved from large infrequent surveys to daily Voice of the Employee forms capturing real-time user sentiment."

"Every minute spent struggling with an IT system is a minute not delivering value to IBM and our clients."

"Don’t try to control or force data to say what you want. The goal is for data to help make the right decision."

"Measurement matters. Even from a small circle, you can make a big impact."

Ask the Rosenbot
Ariba Jahan
Team Resiliency Through a Pandemic
2024 • DesignOps Summit 2020
Gold
Emily DiLeo
Stronger Together: Lessons Learned from UX Research Ops
2024 • DesignOps 2024
Gold
Ana Ferreira
Designing Distributed: Leading Doist’s Fully Remote Design Team in Six Countries
2024 • DesignOps Summit 2020
Gold
Jane Reid
Self-care in User Research (Videoconference)
2020 • Advancing Research Community
Bria Alexander
Opening Remarks
2022 • Civic Design 2022
Gold
Victor Udoewa
Beyond Methods and Diversity: The Roots of Inclusion
2024 • Advancing Research 2024
Gold
Bria Alexander
The Big Question about Resilience: A panel discussion
2024 • DesignOps 2024
Gold
Jemma Ahmed
Theme 2 Intro
2024 • Enterprise Experience 2020
Gold
Lada Gorlenko
Theme 1: Intro
2024 • Enterprise Experience 2020
Gold
Mike Oren
Why Pharmaceutical's Research Model Should Replace Design Thinking
2023 • Advancing Research 2023
Gold
Carol Smith
Operationalizing Responsible, Human-Centered AI
2023 • Enterprise UX 2023
Gold
Farid Sabitov
Automatization for Large Enterprise Teams
2024 • DesignOps Summit 2020
Gold
Anna Avrekh
Expert Panel: Leading in and with Research
2022 • Advancing Research 2022
Gold
Jennifer Kong
[Case study] Journeying toward AI-assisted documentation in healthcare
2024 • Designing with AI 2024
Gold
Chris Geison
What is Research Strategy?
2021 • Advancing Research 2021
Gold
Sean Dolan
A Practical Look at Creating More Usable Enterprise Customer Journeys (Videoconference)
2019 • Enterprise Community

More Videos

Zariah Cameron

"A kickback is a free flowing and life giving gathering that transforms and electrifies throughout the night."

Zariah Cameron

ReDesigning Wellbeing for Equitable Care in the Workplace

September 23, 2024

John Cutler

"Ownership means belonging to the person who anticipates, observes, or monitors the prioritization process—not distributed ownership."

John Cutler Harry Max

Prioritization for designers and product managers (1st of 3 seminars) (Videoconference)

June 13, 2024

Nick Cochran

"Treat every conversation as an opportunity to learn and to make connections."

Nick Cochran

Growing in Enterprise Design through Making Connections

June 3, 2019

Deanna Smith

"Set measurable goals connected to the specific problem to track if a change is effective."

Deanna Smith

Leading Change with Confidence: Strategies for Optimizing Your Process

September 23, 2024

Magdalena Zadara

"Meeting stakeholders where they are means understanding their language and prior knowledge, not just the physical location."

Magdalena Zadara

Zero Hour: How to Get Far Quickly When Starting Your Digital Service Unit Late

November 16, 2022

Nova Wehman-Brown

"Be the one who picks up the sharpie first, especially if you’re in a leadership position."

Nova Wehman-Brown

We've Never Done This Before

June 4, 2019

Liwei Dai

"Artificial intelligence is about building human-like intelligence. This has opened up a golden opportunity for all of us to make a big impact."

Liwei Dai

The Heart and Brain of the AI Research

March 31, 2020

Gordon Ross

"There were quick and dirty solutions whose legacy remains a risk for ongoing COVID work, not all a good news story as we pivot out of COVID."

Gordon Ross

12 Months of COVID-19 Design and Digital Response with the British Columbia Government

December 8, 2021

Tutti Taygerly

"We use design thinking to make north stars for products; I help people find their north stars."

Tutti Taygerly

Make Space to Lead

June 12, 2021