Log in or create a free Rosenverse account to watch this video.
Log in Create free account100s of community videos are available to free members. Conference talks are generally available to Gold members.
Integrating Qualitative and Quantitative Research from Discovery to Live
Summary
Over the last three years Southampton University in the UK has been doing a complete website redesign following an Agile process; user research and performance analytics have been an integral part of the Agile process from the very start. The website has a variety of products aimed at potential students and research collaborators. Performance analytics informed the business case and objectives and qualitative research in Discovery uncovered the user needs to improve the user experience. During alpha qualitative research informed the design of the early prototypes and in beta analytics and user research integrated qual and quant in a variety of metrics around performance and the user experience. The same metrics have been taken forward and enhanced in Live to ensure continuous improvement which sits alongside the new product roadmap. The presentation will outline the integration of qual and quant and give examples of what has been done, the metrics, and how they are informing the user experience and business objectives.
Key Insights
-
•
The University of Southampton faced over 4 million URLs due to decentralized content, causing poor user experience and high bounce rates.
-
•
Replacing 1000+ web content authors with a skilled specialist team improved content quality, consistency, and sustainability.
-
•
Integrating quantitative analytics with qualitative user research enabled a comprehensive understanding of user journeys and needs.
-
•
Measuring over 30 distinct user journeys allowed targeted benchmarking and direct evaluation of site performance.
-
•
Adopting user-centered design was a cultural shift from the university’s previous waterfall approach.
-
•
The beta phase attracted over 4,000 users, providing rich data to validate design choices before full launch.
-
•
Heat maps and recordings revealed user behaviors like 'click rage' where users clicked on non-clickable elements.
-
•
Microsoft reaction cards helped capture user sentiment efficiently, complementing free-text feedback.
-
•
Continuous improvement is enabled by automated data analysis using R scripts and a process to triage issues for design sprints or backlog.
-
•
Stakeholder involvement in observing real user sessions is crucial to overcoming resistance and building empathy for user-centered design.
Notable Quotes
"We are not collecting vanity analytics, the data is doing real work on informing design."
"85% of people were leaving after viewing only one page, showing severe bounce issues."
"The new site had a 19% dropout rate compared to 32% on the old site, showing clear improvement."
"User journeys are the bedrock for deciding which data to collect and measure."
"Some stakeholders still prefer their beliefs to data, even in academia."
"Heat maps showed users clicking on areas they thought were clickable but weren’t, resulting in click rage."
"We brought all the user journeys together with UX designers, content designers, product owners — it wasn’t just analytics and research in isolation."
"Having the qualitative and quantitative data together lets us tell a comprehensive story to stakeholders."
"We started analytics at the beginning so we could benchmark and compare old and new site performance."
"Agile can feel like a cult, but the data helps show it’s working and not just a belief system."
Or choose a question:
More Videos
"At the turning point, the team realized they were in charge of their own process and got engaged."
Carl TurnerYou Can Do This: Understand and Solve Organizational Problems to Jumpstart a Dead Project
March 28, 2023
"If you have somebody on your team for three years, for five years, that’s amazing from a longevity perspective given enterprise legacy and complexity."
Karen PascoeDeveloping Experience Teams and Talent in the Enterprise
June 8, 2016
"Support software used to be really shitty and expensive, and Zendesk set out to fix that by making it beautifully simple."
Jilanna WilsonDistributed Design Operations Management
October 23, 2019
"We’ve seen self-service tool usage grow from hundreds to millions of veterans served monthly after launching a unified VA platform."
Shawna Hein Kevin HoffmanCreate a Cohesive Civic Design Practice Across Agency, Vendors, and Contracts
November 17, 2022
"Site visits with users don’t just give actionable insight; they give us raw, humbling inspiration to keep going."
Barb SpantonDoing Work That Matters: A Look Beyond The Idealistic Notion of 'Doing Meaningful Work'
June 10, 2022
"We decided not to chase shiny objects because we wanted to stay true to our mission."
Briana ThomasThe Quiet Force: Uncovering Hidden Leadership in High-Impact Design Teams
September 24, 2024
"Therapists create space for clients to reflect and discover insights in a safe environment."
Gina MendoliaTherapists, Coaches, and Grandmas: Techniques for Service Design in Complex Systems
December 3, 2024
"Now everyone’s an edge case, so research needs to focus on those edge cases, not just the average user."
Lija Hogan Milan Mijatovic Sam Proulx Louis RosenfeldThree Years Out: Perspectives on the Near-Term Future of User Research
March 15, 2024
"With our new operating model, partners are more confident funding UX ops because of increased clarity and focus."
John Calhoun Rachel PosmanTwo Sides of the DesignOps Coin: Teams Ops and Product Ops
January 8, 2024
