Skip to content

When the Numbers Stopped Making Sense: How I Learned to Listen Again — A UX Researcher’s Story

A UX researcher shares a powerful realization: AI can extract data, but it takes human listening to truly understand users. Discover how she pivoted from pure numbers to reconnecting with real user stories, transforming her approach to product development.

When the Numbers Stopped Making Sense: How I Learned to Listen Again — A UX Researcher’s Story

I used to believe data was the ultimate truth. As a user experience researcher, I spent years designing surveys, running focus groups, and analyzing heatmaps. I thought if I could just gather enough numbers, I would understand what users really wanted. But then I started using AI to conduct user interviews at scale — and everything I thought I knew began to unravel.

Ad for Prompt Pub - Prompts for professionals

It began innocently enough. A colleague suggested we try an AI-powered tool that could interview hundreds of users simultaneously, transcribe their responses, and convert qualitative quotes into quantifiable data points. “It’s efficient,” they said. “You’ll get insights faster than ever.” I was skeptical, but curious. So I ran a pilot.

The results were staggering — at first. Charts bloomed across my dashboard. Sentiment scores climbed. Word clouds revealed “easy,” “fast,” and “intuitive” as top themes. I presented the findings to stakeholders. Everyone nodded. We celebrated. We made product changes based on those numbers.

Then I visited a local community center to observe users interacting with our redesigned interface. I watched a woman in her 60s struggle to complete a task that, according to the AI, “92% of users found intuitive.” She muttered under her breath, “It’s not that I don’t understand — it’s that it doesn’t understand me.” I sat beside her. She told me she had tried three times to reach customer support before giving up. The AI had not captured that. It had not captured her frustration, her resignation, her quiet dignity.

That was my pivotal moment.

I realized the AI was not listening — it was extracting. It was turning human stories into data points, stripping away context, emotion, and nuance. It was mistaking volume for validity. And I had let it happen.

I could not unsee what I had seen. I could not unhear what I had heard. So I did something radical: I stopped relying on the AI for user insights. Instead, I started a community initiative called “Voices Unfiltered.”

The goal was simple: to bring back the human element in user research. We recruited volunteers — not just from our user base, but from local libraries, senior centers, schools, and community organizations. We trained them to conduct in-person interviews using open-ended questions. We recorded the sessions — not for AI analysis, but for human reflection. We created a shared space where researchers, designers, and developers could listen to the raw audio, read the transcripts, and discuss what they heard.

We did not eliminate technology. We repurposed it. We used AI to help us organize and tag interviews — not to interpret them. We used analytics to identify patterns — but only after we had first listened deeply. We built a feedback loop where users could see how their words shaped the product. We even invited some of them to sit in on design sprints.

The impact was immediate. Our product team began asking different questions. Instead of “How can we make this faster?” they asked, “Why does this feel confusing to some users?” Instead of “What feature gets the most clicks?” they asked, “What does this feature mean to someone who has never used technology before?”

One of our most powerful moments came when a young mother, who had participated in our initiative, shared her story during a company-wide meeting. She said, “I didn’t think my voice mattered. But when I saw the changes you made — because of what I said — I felt seen. And that’s worth more than any rating.”

We are not done. We are still learning. But we are listening — truly listening — again.

If you are using AI to understand people, I urge you to pause. Ask yourself: Who is being left out of the data? What stories are being flattened? What emotions are being ignored? Technology can scale. Humanity cannot be scaled. It must be cultivated — one conversation at a time.

I used to think data was the truth. Now I know: truth is in the voice. In the pause. In the sigh. In the story that cannot be reduced to a number.

And that is why I listen — not for answers, but for understanding.


More from The Editor

See all