Summary
Research and analytics are the eyes and ears of DesignOps. Validating whether strategies are effective or ineffective allows course correction and adaptation. Identifying problems that require action is essential for facilitating collaboration and ensuring that team members are engaged. Prioritizing user needs is the heart of user experience and Agile development. None of this can be done well without an effective research program. We’ll share how we scale research efforts to collect feedback from thousands of users per month and track the impact of dozens of projects involving 120+ designers.
Key Insights
-
•
IBM’s CIO IT department is only 3% of the company but manages IT for 12,000 IBMers; the CIO design team is just 1% of that group.
-
•
Over half of IBM’s 385,000 employees have been hired within five years, introducing ongoing challenges in onboarding and experience design.
-
•
30% of IBMers work remotely or offsite, necessitating resilient, intuitive digital tools without onsite support.
-
•
The team uses an IT pyramid of pain inspired by Maslow’s hierarchy to ensure foundational tech and network reliability before focusing on tasks.
-
•
Continuous measurement includes tracking NPS, goal completion, ease of use, and user comments linked directly to employee IDs for rich insights.
-
•
IBM shifted from infrequent large surveys to daily embedded Voice of the Employee forms capturing immediate feedback on 4,000 submissions/month.
-
•
The design ops team integrates research, UX, content, and coaching to foster collaboration and prevent silos.
-
•
A predictive AI model helps identify when employees should upgrade their hardware before critical failures occur, leading IBM to offer on-demand new machines.
-
•
Transparency in metrics drives improvement and accountability, with NPS trending to be incorporated into employee evaluations starting next year.
-
•
Kristen emphasizes embracing chaos and iteration in design ops rather than forcing rigid processes, highlighting the team's evolution from phase one simplicity to complex maturity.
Notable Quotes
"This small but mighty CIO design team is literally 1% of the 3% IT population."
"Over half of IBMers have been here less than five years, adding complexity to designing their experience."
"The state of IT is a daily reflection of what IBM thinks and feels about its employees."
"You can’t just focus on the task level, you have to consider the entire ecosystem and its layers."
"People take out a pen when the CIO says I have a design user experience and research function reporting directly to me."
"We’re here to make work better, and the coaching tribe is here to make teams better."
"We moved from large infrequent surveys to daily Voice of the Employee forms capturing real-time user sentiment."
"Every minute spent struggling with an IT system is a minute not delivering value to IBM and our clients."
"Don’t try to control or force data to say what you want. The goal is for data to help make the right decision."
"Measurement matters. Even from a small circle, you can make a big impact."
Dig deeper—ask the Rosenbot:
More Videos
"Amazon performs better on mobile than on desktop according to our Net Promoter Score measurements."
Daniela Magaña Flores Ariane Rahn Jeff Ephraim BanderAhead of Competition: Learn What UX Benchmarking Can Do for Your Business Today
March 10, 2022
"When humans blindly prompt a large language model, they’re not receiving wisdom from an all-knowing oracle but probabilistic outcomes based on patterns."
Savina HawkinsHarnessing AI in UXR: Practical Strategies for Positive Impact
March 26, 2024
"We are witnessing the most rapid loss of biodiversity in human history."
Alex Hurworth Bonnie John Fahd Arshad Antoine MarinDesigning a Contact Tracing App for Universal Access
October 23, 2020
"Building a custom repository is resource intensive, but offers control necessary for large organizations like Microsoft."
Matt DuignanHITS, Microsoft's internal human insight system: From research library to living body of knowledge (Videoconference)
July 16, 2019
"Make it easy to do the right thing, and as a side effect, make it harder to do the wrong thing - guiding principle from Salesforce Lightning Design System."
Craig VillamorDesign Systems for Ethical Design (Videoconference)
January 26, 2023
"Think of UX as risk mitigation, not a zero-sum game, and share the risk with your stakeholders."
Ovetta SampsonTurning UX Passion into Real Product Influence
June 7, 2023
"UX assets are our business assets. How do we talk about our UX assets as business assets?"
Rima Campbell Amrit S BhachuIncrease Productivity and Drive Business Impact
September 24, 2024
"Design ops teams are mostly a team of one, so many of you are probably in the same boat I was."
Amy Gawronski ZuccaroAdvice for DesignOps Employee #1
September 29, 2021
"Songs are our way of knowing; you can tell someone's emotional state by the song they sing."
Victor UdoewaBeyond Methods and Diversity: The Roots of Inclusion
March 26, 2024
