Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Research Operations at Scale

Gold
Tuesday, November 7, 2017 • DesignOps Summit 2017
Share the love for this talk
Research Operations at Scale
Speakers: Christian Rohrer
Link:

Summary

Research is an inherent part of design, and operationalizing how insights are generated and used is one of the more challenging aspects of design operations. Overcoming the lack of good tools, taming the complexity of research at scale, and navigating the world of different forms of inquiry are just a few of the obstacles to overcome. That said, we are starting to see a range of benefits from implementing research operations in modern design organizations.

Key Insights

  • Banks like Capital One must manage research amid strict regulations and risk, requiring innovative recruiting and study designs.

  • Specialized research operations and curricula teams at Capital One remove procedural inefficiencies and ensure compliance.

  • Risk Management at Capital One includes embedded 'risk-sharper' roles that guide research teams through compliance challenges.

  • 'Wine and dine' sessions enable rich qualitative insights by combining customer interaction with large employee participation, without traditional focus group pitfalls.

  • User research approaches must vary by business context: B2C, B2B, and internal IT differ greatly in user access and needs.

  • Agile’s dominance in software development creates challenges for user-centered design integration, requiring adaptations or use of Lean methodologies.

  • Qualitative research provides rich user insight and inspiration for designers, while executives rely more on quantitative data for decision-making.

  • The PURE method formalizes usability evaluation through expert ratings, producing measurable scores that drive executive action and product improvement.

  • The user experience talent pool is small and highly sought after, making it critical to structure roles to attract and retain specialists rather than expecting 'unicorns.'

  • Effective knowledge management for UX research must support the entire insight lifecycle—from ideation to product impact—to avoid losing valuable learnings.

Notable Quotes

"Capital One is really a bank ... it’s out to change banking for good by becoming a tech company that just happens to do banking."

"We call it research goo: regulation, risk, inefficiencies that complicate research in banks."

"Risk-sharper is a role assigned within Risk Management that carries teams through the risk process to mitigate it."

"Wine and dine is not a focus group; it’s more employees than customers interacting in small groups to get deep insights."

"In business-to-business contexts, finding a champion within your sales or account management teams can open access to users for research."

"You need to know tomorrow’s user as well as today’s because their characteristics differ greatly according to diffusion of innovation."

"Agile is here to stay because engineering managers won’t hire for anything else; user-centered design must fit into Agile somehow."

"Designers thrive on qualitative research because it gives them access to the root of the problem beyond just numbers."

"Never put numbers on qualitative research findings; qualitative insights are about richness and why, not how many."

"The PURE method measures usability by scoring cognitive load and friction at each step of fundamental user tasks."

Ask the Rosenbot
Ian Swinson
Designing and Driving UX Careers
2016 • Enterprise UX 2016
Gold
Shan Shen
Translating UX Terms into Business Contexts
2023 • Design in Product 2023
Gold
Prayag Narula
Empowering Designers to do Good Research
2022 • Advancing Research 2022
Gold
Shipra Kayan
How Tess Dixon Facilitates Team Engagement and Collaboration at Condé Nast Using Miro 
2021 • DesignOps Summit 2021
Gold
Peter Boersma
How to Define and Maintain a DesignOps Roadmap
2023 • DesignOps Summit 2023
Gold
Sarah Auslander
Insights Panel
2022 • Civic Design 2022
Gold
Jake Burghardt
Finding More Inroads into Research Impact
2026 • Rosenfeld Community
Anne Mamaghani
How Your Organization's Generative Workshops Are Probably Going Wrong and How to Get Them Right
2023 • Advancing Research 2023
Gold
B. Pagels-Minor
Breaking the Tension: The Power of Enabling Your Employees to Show Up Authentically
2022 • Design at Scale 2022
Gold
Ilana Lipsett
Anticipating Risk, Regulating Tech: A Playbook for Ethical Technology Governance
2021 • Civic Design 2021
Gold
Leah Buley
Ask Me Anything with Leah Buley and Joe Natoli, co-authors of The User Experience Team of One (2nd edition)
2024 • Rosenfeld Community
Laura Klein
Human vs. machine: Testing AI’s ability to synthesize and analyze research
2026 • Advancing Research 2026
Gold
John Calhoun
Two Sides of the DesignOps Coin: Teams Ops and Product Ops
2024 • DesignOps Summit 2020
Gold
Rima Campbell
Increase Productivity and Drive Business Impact
2024 • DesignOps Summit 2024
Gold
Nalini P. Kotamraju
An Organizational Story: Salesforce Lightning Design System
2016 • Enterprise UX 2016
Gold
Sam Proulx
Designing For Screen Readers: Understanding the Mental Models and Techniques of Real Users
2021 • Civic Design 2021
Gold

More Videos

John Cutler

"Maturity is not linear; the tanker company had a lot to learn from the speedboat, and vice versa."

John Cutler

Oxbows, Rivers, and Estuaries: How to navigate the currents of change (without burning out)

December 3, 2024

Darian Davis

"If you’re generating buy-in, don’t forget what it will mean for your difficult stakeholders—they may have external pressures you don’t know about."

Darian Davis

Lessons from a Toxic Work Relationship

January 8, 2024

Dave Gray

"Our communications is all over the map — email, Slack, Hangouts, face-to-face, GitHub — it surfaced some gaps we didn’t fully appreciate before."

Dave Gray

Group Activity: Making Sense of DesignOps

November 7, 2017

John Cutler

"The subtle art of taking care of yourself and giving a f in ways that generally encourage coherent behaviors."

John Cutler

The Alignment Trap

November 29, 2023

Abby Covert

"Diagrams help when we feel stuck—providing stability, transparency, understanding, clarity, and kindness."

Abby Covert

Stuck? Diagrams Help

October 27, 2022

Mark Interrante

"Culture is what you tolerate — if you tolerate rudeness or tardiness, it becomes part of who you are."

Mark Interrante

Collaboration Flows in Product Development

June 9, 2017

Devon Powers

"If I do a project today on 2025, you’re not going to be in that chair in 2025 and neither am I."

Devon Powers

Imagining Better Futures

March 9, 2022

Prayag Narula

"There is overlap between design and research when it comes to different types of research; it’s not a strict divide."

Prayag Narula

How to Empower Your Designers to Do Good Research – And Why You Want To

June 10, 2022

Peter Van Dijck

"You can’t just be a designer adapting UX workflows assuming everybody else’s workflow stays the same; everything is changing with AI."

Peter Van Dijck

Hands on AI #3: Claude Code for UX people

October 22, 2025