Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Integrating Qualitative and Quantitative Research from Discovery to Live

Friday, December 16, 2022 • QuantQual Interest Group
Share the love for this talk
Integrating Qualitative and Quantitative Research from Discovery to Live
Speakers: Mackenzie Cockram , Sara Branco Cunha and Ian Franklin
Link:

Summary

Over the last three years Southampton University in the UK has been doing a complete website redesign following an Agile process; user research and performance analytics have been an integral part of the Agile process from the very start. The website has a variety of products aimed at potential students and research collaborators. Performance analytics informed the business case and objectives and qualitative research in Discovery uncovered the user needs to improve the user experience. During alpha qualitative research informed the design of the early prototypes and in beta analytics and user research integrated qual and quant in a variety of metrics around performance and the user experience. The same metrics have been taken forward and enhanced in Live to ensure continuous improvement which sits alongside the new product roadmap. The presentation will outline the integration of qual and quant and give examples of what has been done, the metrics, and how they are informing the user experience and business objectives.

Key Insights

  • The University of Southampton faced over 4 million URLs due to decentralized content, causing poor user experience and high bounce rates.

  • Replacing 1000+ web content authors with a skilled specialist team improved content quality, consistency, and sustainability.

  • Integrating quantitative analytics with qualitative user research enabled a comprehensive understanding of user journeys and needs.

  • Measuring over 30 distinct user journeys allowed targeted benchmarking and direct evaluation of site performance.

  • Adopting user-centered design was a cultural shift from the university’s previous waterfall approach.

  • The beta phase attracted over 4,000 users, providing rich data to validate design choices before full launch.

  • Heat maps and recordings revealed user behaviors like 'click rage' where users clicked on non-clickable elements.

  • Microsoft reaction cards helped capture user sentiment efficiently, complementing free-text feedback.

  • Continuous improvement is enabled by automated data analysis using R scripts and a process to triage issues for design sprints or backlog.

  • Stakeholder involvement in observing real user sessions is crucial to overcoming resistance and building empathy for user-centered design.

Notable Quotes

"We are not collecting vanity analytics, the data is doing real work on informing design."

"85% of people were leaving after viewing only one page, showing severe bounce issues."

"The new site had a 19% dropout rate compared to 32% on the old site, showing clear improvement."

"User journeys are the bedrock for deciding which data to collect and measure."

"Some stakeholders still prefer their beliefs to data, even in academia."

"Heat maps showed users clicking on areas they thought were clickable but weren’t, resulting in click rage."

"We brought all the user journeys together with UX designers, content designers, product owners — it wasn’t just analytics and research in isolation."

"Having the qualitative and quantitative data together lets us tell a comprehensive story to stakeholders."

"We started analytics at the beginning so we could benchmark and compare old and new site performance."

"Agile can feel like a cult, but the data helps show it’s working and not just a belief system."

Ask the Rosenbot
Ned Dwyer
Right horses for the right courses – how and when to democratize research
2025 • Advancing Service Design 2025
Gold
Andrea Gallagher
The Problem Space
2019 • Advancing Research Community
Erin Hoffman-John
This Game is Never Done: Design Leadership Techniques from the Video Game World
2017 • DesignOps Summit 2017
Gold
Ron Bronson
Design, Consequences & Everyday Life
2022 • Civic Design 2022
Gold
Erin May
Distributed, Democratized, Decentralized: Finding a Research Model to Support Your Org
2022 • Advancing Research 2022
Gold
Craig Brookes
"Just Make it Look Good" and Other Ways We're Misunderstood
2021 • Design at Scale 2021
Gold
Alfred Kahn
A Seat at the Table: Making Your Team a Strategic Partner
2023 • Design in Product 2023
Gold
Taylor Klassman
Shaping the Next Era of UX Research: Collaborative Forum
2025 • Advancing Research 2025
Gold
Greg Petroff
Everything is About to Change: Software as Material
2016 • Enterprise UX 2016
Gold
Angelos Arnis
State of DesignOps: Learnings from the 2021 Global Report
2021 • DesignOps Summit 2021
Gold
Laura Klein
Human vs. machine: Testing AI’s ability to synthesize and analyze research
2026 • Advancing Research 2026
Gold
Greg Petroff
Software as Material—A Redux
2023 • Enterprise UX 2023
Gold
Christian Crumlish
Morning Insights Panel
2022 • Design in Product 2022
Gold
Noah Bond
Redefining truth and inclusivity: Navigating data ownership and ethical research in the age of disinformation
2025 • Advancing Research 2025
Gold
Jesse Zolna
Inviting the Whole Org to Come See For Yourself
2020 • Advancing Research 2020
Gold
Jerome “Axle” Brown
How to Use Self-Directed Learning to Ensure Your Research Insights are Heard and Acted Upon
2021 • Advancing Research 2021
Gold

More Videos

Xenia Adjoubei

"The Researcher in Residence program invites members of affected communities to use our tools over a funded three-month period pro bono."

Xenia Adjoubei Sean Bruce

Empowering Communities Through the Researcher in Residence Program

March 29, 2023

Tamara Kartoziia

"Our framework is our playbook, our single source of truth for every launch."

Tamara Kartoziia

Think global, adapt local: how service design accelerated B2B market entry by 6 months

November 20, 2025

Jay Bustamante

"Rushing to market with AI solutions can irreparably damage not only your product but your entire brand."

Jay Bustamante

Navigating the Ethical Frontier: DesignOps Strategies for Responsible AI Innovation

October 2, 2023

Nicole Aleong

"Not everyone has the same capacity to aspire to a future; this is critical when working with marginalized communities."

Nicole Aleong

Future Orientations to Everyday Life: Futures Anthropology as a Methodology

March 26, 2024

Nick Lewis

"Using generative AI for design risks losing the human creativity that makes design special."

Nick Lewis

Designing and building low-carbon websites independently

November 18, 2025

Vicky Teinaki

"We’ve enabled exposure hours for our entire teams, taking the heat off our recruiters."

Vicky Teinaki Michele Marut Tim Parmee

Short Take #3: UX/Product Lessons from Your Industry Peers

December 6, 2022

Laura Weiss

"The alternate to engaging with conflict is suffering quietly or accumulating conflict debt."

Laura Weiss

Turn Down the Heat: 3 Ways to Handle Conflict in the Moment

November 20, 2024

Joerg Beringer

"The tool enables designers and product managers to generate high-quality context and user requirements without dependency on user access or research skills."

Joerg Beringer Thomas Geis

Scaling User Research with AI: Continuous Discovery of User Needs in Minutes

June 10, 2025

Shelby Switzer

"Silence is productive—resist the temptation to fill it immediately after asking a question."

Shelby Switzer

Making Space for Community Knowledge-sharing in a Distributed World

December 10, 2021