Why scientific rigor matters in UX research, and how to bring it to fast-moving teams

Dr. Mohsen Rafiei, Assistant Professor of Psychology at the University of Arkansas atLittle Rock (UALR) and Quantitative UX Research Lead at the Perceptual User Experience Lab (PUXLab), has lived on both sides of this equation. First, as an experimental scientist, and later as an applied researcher for global brands such as Adidas.

As he explains: “In academia, we usually have months to dive into a question. But in industry it’s a whole different game... you’ve got multiple managers to align with, and what they want is results in two weeks with five participants...”

The challenge, then, is clear: how do UX research teams uphold scientific rigor when time and resources are limited?

The role of rigor in UX research

For Dr. Rafiei, rigor is not optional; it is what makes research meaningful. “Doing UX research without a solid scientific foundation (without being trained to apply scientific rigor) is like trying to perform surgery without knowing how. Sure, you might be busy with scalpels and gloves, but no one’s walking out of that operating room better off... In research, it’s the same, you might run interviews, crunch numbers, or make fancy graphs, but if rigor isn’t there, the result isn’t valuable. At best, it’s noise dressed up as insight…. At worst, it’s a wrong conclusion that leads a whole team in the wrong direction.”

At Rally, we see rigor not as a drag on velocity but as the factor that makes even small, scrappy studies worth doing. Without it, insights become noise. With it, even five-person studies can help teams avoid costly mistakes.

What UX researchers bring to the table

UX researchers must be able to safeguard rigor under real-world constraints. As Dr. Rafiei explains: "You don’t become a researcher through a quick bootcamp. It usually takes many years to learn how to design studies, control for unwanted variables, and choose the right methods for analyzing data. UX researchers must also understand human behavior, recognize the biases that appear in different types of data, and know how to account for them so the results are meaningful. That level of training and experience is what makes research insights trustworthy and ultimately useful for guiding decisions."

Professional training and experience gives researchers specialized skills:

• Selecting the right methods for small samples
  “If you’re working with fewer than 10 participants, running a parametric test usually isn’t the best choice because the assumption of normality is shaky. In those cases, it’s better to lean on methods that are more appropriate for small samples.” Trained UX researchers know how to weigh the options and pick the test that makes sense given the data and resources.
• Reducing bias in study design
  Whether it’s piloting a study or using within-subject controls, trained UXRs learn strategies to reduce bias and cut down on unnecessary noise.
• Interpreting statistics responsibly
  As Dr. Rafiei explains, “managers don’t really care if you’re using a t-test, ANOVA, Wilcoxon, or bootstrapping. What they care about is getting insights they can trust. It’s the researcher’s responsibility to choose the method that gives the most meaningful and reliable result for the situation at hand.”

These skills ensure that quick-turn studies still generate insights that teams can trust.

Research as a team sport

Still, Dr. Rafiei emphasizes that while researchers bring the rigor, they don’t work in isolation. PMs, designers, and engineers all play essential roles in shaping the direction of a project. But when it comes to how the research is carried out, trust is key. As he puts it, “As a leader or manager, you can set the goals and boundaries for UX research, but you have to trust researchers to do the work in the way that ensures the insights are reliable and valuable.”

At Rally, we agree: research is a collaborative process. Non-researchers contribute domain expertise, business context, and the ability to apply findings. Researchers contribute the scientific foundation that ensures those findings are valid. The strongest outcomes emerge when each role respects the others’ expertise.

Tactical advice for enterprise teams

Dr. Rafiei shared practical strategies for researchers facing small samples and short timelines.

Working with small sample sizes

• “If you have fewer than 10 participants, parametric tests usually aren’t the best choice,” he notes. In those situations, non-parametric methods such as Wilcoxon or Kruskal-Wallis are often more appropriate, and Bayesian approaches can be useful if priors are available.

Recruitment tips

• “With a small N, every participant carries more weight,” says Rafiei. It’s especially important to recruit people who are genuinely representative of the domain, language, and context you’re studying.

Study design strategies

• “Within-subject designs are often more efficient and reliable than between-subjects when N is small. Running a pilot, even a quick one, can also help catch issues early.”
• “Running a pilot, even a quick one, can also help catch issues early.”

Framing and communication

• Researchers should be upfront about uncertainty and limitations. And stakeholders, who usually aren’t trained researchers, should trust UXRs’ judgment on how the study is conducted. As Rafiei puts it: “They should trust researchers to decide how to do the research.”

Rigor as a force multiplier

At the end of the day, rigor is not about slowing teams down. It is about making sure that every research effort, whether small or scrappy, has the best chance of steering the business in the right direction.

As Dr. Rafiei sums up: "Hire a trained, well-educated researcher. A skilled researcher can evaluate competing designs and determine which one is most effective, which one truly delivers value. That expertise means the business is not gambling on intuition or personal opinion, but making decisions grounded in evidence. In the long run, this prevents wasted resources, reduces costly missteps, and ensures that investments are directed toward solutions that actually work."

For enterprise leaders, the takeaway is clear: research is a collaborative effort, but rigor is what multiplies its impact. By investing in trained researchers, respecting their expertise, and giving them the space to do their work, organizations gain insights that are both credible and actionable. Even when studies are lean and based on small samples, rigor ensures that the findings can guide strategy with confidence. The outcome is not just informed decisions, but decisions that pay off in meaningful, measurable ways.

Dr. Rafiei concluded with a clear message: “If we want UX to remain practical and functional in industry, in a way that earns genuine trust, we must bring rigor back into it. Without rigor, research may look polished but fail to deliver lasting value. With rigor, UX becomes a reliable partner for guiding decisions, shaping strategy, and demonstrating its true impact.”