Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Scaling Research Via an Ops First Model at Clever

Gold
Monday, March 27, 2023 • Advancing Research 2023
Share the love for this talk
Scaling Research Via an Ops First Model at Clever
Speakers: Liza Pemstein and Jane Davis
Link:

Summary

In this Q&A session, we'll hear how Liza Pemstein, Research Operations Manager at Clever, built a robust research practice at one of the fastest-growing EdTech companies. Identifying the friction in the process experienced by design, product, engineering, and marketing, Liza has streamlined and scaled research. She'll share how she got started, what she's learned and what she's excited about in 2023!

Key Insights

  • Clever operates a research operations first model with no dedicated UX researchers, relying on designers and PMs to conduct research.

  • Liza Pennstein’s path to research operations included customer solutions roles emphasizing empathy, listening, and problem-solving.

  • Centralizing participant incentives through a shared credit card and Jira ticket system eliminated confusion and improved process efficiency.

  • Recruiting and scheduling user research participants was a major barrier that was addressed by creating shared participant databases.

  • The ROCKS program at Clever (Research Ops Knowledge Share) facilitates cross-team learning and un-silos research knowledge.

  • Initial focus was on increasing research confidence and comfort among practitioners before emphasizing quality and best practices.

  • Balancing standardization with flexibility is critical; intervention happens when research quality or participant screening falters.

  • Without dedicated UX researchers, Clever prioritizes enabling non-experts with training, templates, and external expert speakers.

  • Building a lightweight repository with Great Question software supports storing and sharing research insights effectively.

  • Future plans include broadening methods like unmoderated research, increasing visibility, and maintaining momentum through speaker series and workshops.

Notable Quotes

"Research is a sales role at its heart, delivering value to people to drive sustainable growth."

"I had a hunger for process and efficiency once I built a feedback bot to share customer insights."

"We don’t have dedicated user researchers, but designers and PMs conduct most research here."

"People were doing incentives in different ways, using their own cards, which was confusing, so I centralized it."

"The biggest struggle was recruiting and scheduling participants, so we started a shared Google sheet."

"Research is a group project and learning is a group project—you don’t have to be an expert but find those who are."

"I want to encourage excitement about research and gradually mold people into better researchers."

"Sometimes you have to make it messy before it can be clean again — disrupt to improve."

"We have the quantity of research work; now we’re focused on building up the quality."

"Unmoderated research is exciting because it lets us gather insights while freeing people up to be in two places at once."

Ask the Rosenbot
Alnie Figueroa
Teamwork: Strategies for Effective Collaboration with Other Program Management Teams
2022 • DesignOps Summit 2022
Gold
Justin Entzminger
Risk and Reward: How to Diversify the Field of Civic Innovators and Designers
2022 • Civic Design 2022
Gold
Louis Rosenfeld
Opening Remarks
2023 • Design in Product 2023
Gold
Peter Merholz
Design at Scale is People!
2021 • Design at Scale 2021
Gold
Trisha Terhar
Empathizing with the Empowered: Non-Researcher Responses to Democratization
2022 • Advancing Research 2022
Gold
Nathan Curtis
Design Systems for Us: How Many One-Source(s)-of-Truth Are Enough?
2019 • DesignOps Community
Marc Majers
Interrupted UX - Add A Dose of Reality To Usability Testing
2022 • Advancing Research 2022
Gold
Ian Johnson
Latent Scope: Finding structure in unstructured data
2025 • Designing with AI 2025
Gold
Josh Clark
Sentient Scenes and Radically Adaptive Experiences
2025 • Designing with AI 2025
Gold
Ashley Sewall
Exit Interview #5: Designing My Life After Tech
2026 • Rosenfeld Community
Edgar Anzaldua Moreno
Using Research to Determine Unique Value Proposition
2021 • Advancing Research 2021
Gold
Johnny Michaelsen
Measure Behaviors, Not Results
2026 • Rosenfeld Community
Megan Campos
What Did I Miss? The Hidden Costs of Deprioritizing Diversity in User Research
2021 • Advancing Research 2021
Gold
Ignacio Martinez
Fair and Effective Designer Evaluation
2024 • DesignOps Summit 2024
Gold
Bria Alexander
Opening Remarks
2021 • Design at Scale 2021
Gold
Silke Bochat
5 Antifragile Strategies for a DesignOps 2.0
2024 • DesignOps Summit 2024
Gold

More Videos

Taylor Jennings

"Slides are hitting a brick wall; video clips make findings much more interactive and engaging for stakeholders."

Taylor Jennings Joe Nelson Alex Knoll

Repository Retrospective: Learnings from Introducing a Central Place for UX Research

March 9, 2022

Nicole Aleong

"Anticipation involves preparatory actions to manage uncertainty about future conditions."

Nicole Aleong

Future Orientations to Everyday Life: Futures Anthropology as a Methodology

March 26, 2024

Jeff Gothelf

"Adobe’s Kickbox is a fantastic professional development tool, but it doesn’t drive new lines of business."

Jeff Gothelf

Innovation Studios: the Engines of Enterprise Experimentation

May 14, 2015

Saara Kamppari-Miller

"Tools are great, but they do not solve all problems; training and understanding the why behind accessibility is essential."

Saara Kamppari-Miller

DesignOps for Inclusive Design and Accessibility

May 26, 2022

Tess Dixon

"If you don’t have the basics taken care of, infinite promotions and ping pong tables are just noise."

Tess Dixon

C'mon Get Happy

September 29, 2021

Liam Thurston

"All your team wants to do is win at career Bingo or career Snakes and Ladders, hopefully more ladders."

Liam Thurston

Why Your Design Team Is Quitting, And How To Fix It

June 10, 2022

Ian Swinson

"A workshop helps people map their skills, identify gaps, and make specific goals for career jumps."

Ian Swinson

Designing and Driving UX Careers

June 8, 2016

Leisa Reichelt

"One researcher per team for at least three days a week was considered an outrageous luxury at GDS in 2013."

Leisa Reichelt

Opening Keynote: Operating in Context

November 7, 2018

Rachael Greene

"Without a design ops function, teams make their own flavors that dilute brand and accessibility."

Rachael Greene Alison Davis

Building a Design Ops Practice that Really Works (Most of the Time)

October 2, 2025