Mithila Fox, Stack Overflow’s Director of Product Research, joined the Rally team on March 2 for an AMA to discuss her experience building Stack Overflow’s Research function. Mithila covered topics like increasing Research maturity, building relationships with counterparts and stakeholders, and different types of org models for Research.

If you missed it, or want to take another look at what Mithila shared, read our highlights below. If you’d like to watch the full AMA, head to our YouTube channel.

Key Takeaways

🔑  It’s easy to have discussions about diversity and inclusion in research, but specific and real action is key.

🔑  As a researcher, you’ll always have to balance user goals with business goals. You’ll also have the opportunity to put pressure on making goals and KPIs more user-centric.

🔑  Be a squeaky wheel about research — bring it up a LOT. Share often research goals, insights, and progress.

🔑  One of the most underrated and important skills to have as a researcher is being a good storyteller.

🔑  So many User Researchers are massively qualified and have really diverse and impressive backgrounds. It’s critical to have that expertise understood and respected.

🔑  Stop undervaluing yourself and be confident in your own expertise and why you’re here. Doing so will be key to growing the Research function at your org.

Who is Mithila?

When trying to decide what to study at university, Mithila found herself stuck between psychology and computer science. Her plan was to do both and drop one after a year. “During that time, I found this wonderful overlap that is User Experience Research,” she said. “And I thought, this is exactly what I’m interested in.”

Fast forward to now. Mithila has spent the last decade working in Research in Australia, London, and Amsterdam in startups, tech companies, and fortune 50 — not to mention a wide variety of industries from health and beauty to private aviation.

Now she’s the Director of Product Research at Stack Overflow, where she’s spent the last four years building the Research function. “This work has become more and more interesting over time, which is amazing.”

2 lessons from Mithila’s decade in Research

• Nearly everyone needs research. “This has been consistent in nearly every role I’ve had,” she said. “Research is becoming more prevalent and there’s a huge need for it to be understood.”
• So much of a researcher’s role is about understanding our users. “We can do that in any field and in any kind of geography,” she said. “But, it’s important to be mindful of cultural differences when running research.”

{{quote-mithila-1="/cms"}}

The evolution of Research maturity at Stack Overflow

When Mithila joined Stack Overflow four years ago, there was only one researcher. Now there are nearly 12. Not long after she joined, the team added a second researcher, thanks to the groundwork that had been laid before Mithila joined. “That spoke volumes,” she said.

The team at Stack Overflow already had a basic understanding of research and a budding interest in it. They also recognized that incentives were needed to conduct quality research. But there was still a lot of work to do to get to a point where the full impact of research was understood.

“My goal was to grow the maturity and to reposition us from consultants to key strategic partners who can impact not just design decisions, but also shape roadmaps,” said Mithila.

This meant Mithila and her team started prioritizing new and different types of research and putting a greater focus on answering more big-picture questions. Because of this, many of Mithila’s early days at Stack Overflow were very strategy-focused. And it paid off.

“We surfaced insights that could be used to unlock big business decisions,” she said. “This got us great support from leadership and helped us reinforce the value of research from the top down.” By doing this, Mithila and her team were able to:

• prove the value of research
• show what research can bring to the table
• identify how research can shape strategy and the future of Stack Overflow
• get buy-in from leadership
• do more research
• grow the team

The evolution of ReOps at Stack Overflow

“The need for Research Ops really became most apparent when we started our Research Guild (more on this below 👇),” said Mithila. “We were starting to really see how much we were duplicating work and effort and how much time could be saved with a more consistent, consolidated ReOps process.”

Mithila and her team are currently trying to hire a ReOps person who can handle the day-to-day tactical operations as well as tackle more strategic pieces like how to improve response rates for surveys and how to improve diversity of samples.

She hopes a ReOps professional will

• take a more considered look at everyday recruitment processes
• audit and improve ethical research practices
• enable the team to practice more inclusive research and amplify diverse user voices

“We work in a space that is predominantly male-dominated and that’s not the only voice we want to hear,” said Mithila. “Hopefully, a ReOps professional will help us balance those voices, bring more neurodiverse users to the table, and ensure our processes are inclusive.”

{{quote-mithila-2="/cms"}}\

Mithila vision for the future

“I would love for us to be in a place where we are really able to leverage all the different insights we’re receiving from within Stack Overflow, whether that’s from us, market researchers, data scientists, community managers, etc.,” said Mithila. Mithila also hopes to have a secure place to store those insights that is accessible and enables more decisions to be made using research.

She also wants to continue to grow the Research Guild and keep its collaborative nature central to how the team works. “I hope for us to improve our predictability and be more proactive instead of reactive.”

4 ways to gain support for diversity and inclusion in research

• Reach out to non-traditional stakeholders — especially someone who may specialize or be focused on diversity and inclusion.
• Get feedback from members of ERGs (employee resource group), if your org has them.
• Tie diversity into business goals. “At the end of the day, we need a diverse set of perspectives that are more accurately representative of our user base.”
• Take action. “We can talk a lot about figuring out what should change to be more diverse and inclusion, but specific action is key.”

Building a strong relationship with your stakeholders

“A lot of people have different ideas about exactly what Research is and what User Researchers can do,” said Mithila. It’s important to align with leadership, especially early on. Here’s what Mithila recommends you do:

• Ask the right questions.
• What is your interpretation of Research?
• What are your business goals?
• What are things you care about?
• Start proving your value.

This lays a great foundation for future conversations with leadership. Moving forward, you can now say, ‘I know you’re interested in this topic, and here’s what I found on it. Now here are the decisions it is going to unlock for you.’

“Most leaders who are interested in data-backed decision-making and interested in their audience and growing the business are going to pay attention when you bring something like that to the table,” she said.

Listening to your stakeholders and asking questions like what Mithila listed above will help build a strong foundation for alignment. Before Research can deliver value to the business, it must first deliver value for your users. “You’ll always have to balance user goals with business goals.”

At Stack Overflow, Mithila and her team have worked hard to ensure the voice of their users are understood. “We’ve created many goals that are centered around bringing value to our users,” she said. “Ultimately, Researchers have the opportunity to put pressure on business goals and KPIs and advocate for a more user-centric approach to goal-setting and strategy.”

{{quote-mithila-3="/cms"}}

Mithila also shared four tips for how to bridge the gap between the voice of the user and your stakeholders:

• When possible, bring stakeholders into research sessions.
• Get creative with how you present research insights.
• Share insights as much as possible.
• Bring up research A LOT “be the squeaky wheel about it.”

Bringing stakeholders into research sessions is a great tool, but obviously, Mithila said, it’s only doable with your more immediate collaborators. There are likely many other stakeholders and leaders who just aren’t able to dedicate time to sitting in on research sessions.

That’s why it’s important to think about other people within your org who can advocate for research. If your advocate is not a researcher, Mithila said, think about ways you can support them so they fully understand the value of research and can be a voice for you and your team. Another thing Mithila recommends is to push for proper leadership for your Research function and get Research a seat at the table.

Tips for building better cross-functional relationships

“PM and Designer relationships are always really productive,” said Mithila. But, as with any relationship, there are often challenges. When trying to build strong relationships with PMs, Designers, Data Scientists, and other counterparts, Mithila recommends the following:

• Bring your counterparts, especially Product Managers and Designers, into the planning process early.
• Draw boundaries and make it clear what each person’s expertise or specialty is and how everyone can work together.

Relationships with Data Science

“Data scientists have the best outlook on what data we have available,” said Mithila. “They know what behavioral and usage data we can utilize.” Mithila shared two places to start when trying to build a relationship with your Data Science team.

1. Determine what shared resources you have with the Data Science team.
2. Find commonality in things like skills, interest in data, interest in answering difficult questions, etc.

{{quote-mithila-4="/cms"}}

At Stack Overflow, Mithila’s Researchers are all mixed-methods researchers and many have quantitative backgrounds. “This has proven a key building block for fostering a relationship with the data team,” said Mithila.

Measuring the ROI of Research and making it reusable and accessible

“Measuring ROI can be tricky since User Research is a few steps removed from having a direct impact on revenue or usage behaviors,” said Mithila. But, she added, that doesn’t — and shouldn’t — stop Researchers from focusing on and defining the impact when building research plans or strategies. Identifying the impact means you can then find metrics that are evidence of that impact.

While there are no firm metrics Mithila and her team look at, the most important measure of success for them is that their research is used. “For us, a failed study is one that doesn’t get used,” she said. “As we go forward, we’re really looking to improve the reusability and accessibility of our research results.”

{{quote-mithila-5="/cms"}}

Before research is properly planned out, it’s essential to ensure all stakeholders are aligned on research goals, key research questions you’re trying to answer, and the research impact you want to have, said Mithila. “If people are aligned up front, they know what to expect and can start mentally preparing to use the research.”

Mithila also recommended emphasizing good communication. “One of the most underrated and important skills to have as a researcher is being a good storyteller,” said Mithila. Figure out what style and format your stakeholders care about and what’s going to have the most meaning and impact for delivering research.

{{quote-mithila-6="/cms"}}

Something Mithila and her team are still working toward is being at a place where people can easily access and leverage the wealth of research that they’ve been building over the years. “We’ve tried lots of research repositories, but every solution always has its pitfalls.”

Where should Research sit in an org?

“I’m an advocate for being standalone,” said Mithila. “I don’t think Research nests under Design or Product, though I think they are close collaborators.”

Every org is different and we each have to work within our unique situations, but Mithila said she thinks working toward a standalone Research function should be the goal. “That’s the best opportunity we have to grow in maturity,” she said. “Research is a meaningfully different function with a meaningfully different purpose.”

{{quote-mithila-7="/cms"}}

How does this work at Stack Overflow? Mithila and her team were able to form a Research Guild, which was influential in growing the maturity of their Research function. “I think about what our Research function looks like today versus even two years ago and it’s really considerably different.”

What is Stack Overflow’s Research Guild?

The Guild is a place for researchers across Stack Overflow’s different product lines to come together to support each other, give feedback, improve the quality of research, refine ReOps processes, and grow UX maturity.

“Everyone in the Guild is very motivated to improve research, which ultimately improves the maturity of our research,” said Mithila. “Putting in the effort to bond and share together absolutely pays off.”

Every two weeks the Research Guild meets to discuss current projects and goals. Together, the Guild has completed projects like helping non-researchers run research, defining boundaries and guardrails, and building out other important processes. The Guild also conducts work shares or demo meetings. “It’s an opportunity to share and get feedback on research you’ve done, methodologies, recruitment methods, and in-progress work,” she said.

The Guild’s Slack channel is one of the Research function’s most used channel on Slack. “We’re very lucky that all of our researchers are super engaged,” said Mithila. “It makes for a better space.”

Looking ahead, Mithila and her team are beginning to build up an iInsights Guild, which would be a space meant for anyone interested in research — market researchers, data scientists, product managers, etc. “There are a lot of people who are interested in running various forms of research and by bringing everyone together, we’ll reduce duplication of work, develop and promote a more strategic outlook, and unlock the next level of maturity that we want at Stack Overflow,” said Mithila.

Tips for moving Research into a standalone function

Moving Research out from whatever function it’s under to be a standalone function is tricky, said Mithila. “I’ve had some success in the past with this, and then there have been some places where it just wasn’t going to happen.” Here’s what Mithila recommends you do:

• First, think about the immediate people you can impact with your research. “Your biggest success will come from having senior-level colleagues advocate for research,” said Mithila.
• Second, learn their needs and the big questions they are trying to solve in the product.
• Third, figure out how you can show the value of research and how it’s meaningfully different from the function Research currently sits under.

“Be firm about the value research brings,” said Mithila. “There is a lot online about a soft approach to bringing everyone into the mix and democratizing early. I think there are benefits to that, but there also needs to be an element of putting your foot down and saying no.”

So many User Researchers are massively qualified and have really diverse and impressive backgrounds. “It’s critical to have that expertise understood and respected.”

{{quote-mithila-8="/cms"}}

Restoring clarity to the Research function

A solo researcher in the audience shared this dilemma: Their stakeholders had already been leading research and this researcher was finding it difficult to know how to rein everyone in. “When you’re in this boat, there’s a lot of room for ambiguity,” said Mithila. “Cutting back on that ambiguity wherever possible is really important.”

To do this, Mithila and her team realized they needed to be very clear about the types of research projects that are appropriate for a researcher to take on versus what aren’t. “Drawing this line, wherever it is, is very important.”

Here are some ways you can split up projects:

• type of research (e.g. generative and evaluative research)
• sample size
• where the research is going to be used
• decisions that the research will effect

Along with splitting up projects, it’s essential to have very clear processes and define the division of labor along with individual responsibilities. “With all of these things, you must make sure it’s a collaborative discussion,” said Mithila.

Mithila’s thoughts on AI in UXR

While Mithila doesn’t have firm thoughts on AI in UXR, she said she didn’t see it being anything other than a beneficial tool. Regarding generative AI tools that are emerging, “they seem more like substitutes to help us with things like qualitative coding.” Ultimately, Mithila said, there is no substitute for having a strategic point of view and good, clear storytelling and communication. “Hopefully, AI ends up shortcutting parts of our workflow so that we can still bring that strategic lens to the table.”

{{quote-mithila-9="/cms"}}

Be confident in your value

“As researchers, we often spend a lot of time justifying and explaining our purpose,” she said. “Be confident in your own expertise and why you’re here. It’s time to stop undervaluing ourselves. Once we do so, we’ll be better able to grow the Research functions within our orgs.”

{{quote-mithila-10="/cms"}}

Grow your Research function with Rally

Choosing the right tool can help you to elevate research within your org. At Rally, we’ve built a User Research CRM  that helps teams of all sizes have a centralized view into their participants and allow non-researchers and important cross-functional partners to responsibly take part in User Research. If you want to learn more, join our waitlist!

We are so grateful to Mithila for being willing to share her experience building Stack Overflow’s Research function. If you’d like to watch the full webinar, head to our YouTube channel.