How GitLab streamlined Research Ops and scaled strategic research

In this case study, we’ll explore the challenges the GitLab UX Research team were facing and how they’re using Rally to scale their Research Operations and transform Research from a blocker to a strategic enabler. 

A fast-growing Research team needed infrastructure to keep pace

Fragmented processes, bottlenecks, & stale participant data

Caitlin Faughnan, Senior UX Research Ops Specialist, and her team had outgrown their collection of tools. "Our process had grown organically across multiple tools," explained Caitlin. "We had duplicative tools and we aimed to streamline our processes to align with our scaling."

GitLab's Research teams' biggest need was efficiently accessing user data. GitLab users could only be accessed through one tool that tracked opt-outs and inactive users. The process overall was not as efficient as the team wanted as it took multiple steps involving other GitLab teams. The process to get the data could cause delays and also pull team members from other work.
‍

Coordination & management

"We used a variety of tools and at times it could be difficult to keep track of inbound requests and maintain continuity," said Will Leidheiser, Staff UX Researcher. 

With multiple tools in play, the team sought a more consistent approach to managing outreach. "We wanted a tool that could help with outreach which would help our team spend more time on other initiatives," said Caitlin. The team wanted to also streamline onboarding, again to help the team focus on higher-impact, strategic work.

‍
Sampling consistency & compliance considerations

"We wanted a centralized system serving as a single source of truth to help with sampling and compliance." said Nick Hertz, Senior User Experience Researcher. 

As we have to adhere to respondent sampling rules, we knew we wanted a tool that could help us comply with opt-out preferences, and at times, with a variety of tools in use, it could be time consuming to confirm compliance.

The need for more scalable participant management

We used manual processes to manage participant data and compensation. The team recognized the need for a more automated and scalable approach to participant management to support their growing research program.

GitLab 🤝 Rally’s Research Ops Platform 

Recognizing the need to streamline their processes and reduce the operational burden, Caitlin and her team began searching for a solution and discovered Rally in its early development phase. “We looked at different tools on the market and had early conversations with Rally’s founders about what Rally might look like when it launched,” said Caitlin. “Our search and timing overlapped with Rally going to market.”

During their trial phase, the team experienced firsthand the advantages of Rally. “When we did the trial, we reached out to 14,000 people in our previous tool, and 8,000 in Rally. In the previous tool, we didn’t hit the number we needed, even with multiple waves. In Rally, we hit the number with significantly fewer invites,” she said. “That was huge for us because we have so many people doing research, we can’t afford to waste invitations.”

GitLab’s team quickly realized the value Rally brought to their Research Operations and Research functions. “Once more UXRs got into the tool, they all said they preferred doing research with Rally, whether for surveys or interviews. It was easier to follow and made sense,” Caitlin explained.

“Since adopting Rally, we’ve seen a significant boost in efficiency and effectiveness in the areas we’ve piloted the new process,” said Karen Li, Sr. UX Research Manager. “Previously, a lot of our time was spent on manual tasks. With Rally, we can now streamline participant management, allowing us to run more studies in less time and with fewer errors.”

‍

Here’s how Rally unlocked team-wide Research for GitLab:

‍Direct access to user data - for the first time. 

Rally’s robust Snowflake integration has been particularly transformative, providing the Research team with direct access to user data for the first time. This new access has eliminated the need for manual, multi-step workflows that required involvement from cross-functional partners. “It frees up time on the Product Data Insights team and allows the Research team to recruit users for research significantly faster than before and complete more studies throughout the year,” said Caitlin.

By centralizing research, Rally has made participant management much more straightforward. “In two seconds, I can pay a participant. It’s really easy to track who I’ve interviewed, who I need to pay,” said Will. “I can easily track the participant journey, see if I need to invite more people, or re-invite people who haven’t responded yet.” This streamlined approach has enabled the team to manage participant data more efficiently, significantly cutting down on manual tasks and reducing the risk of errors.

“From a Research Ops and UXR perspective, Rally has helped bring transparency to our outreach process,” Cait added. “Before, the process wasn’t as clear or outlined.” Now researchers have better control over their projects, giving them more time to focus on what truly matters: conducting impactful research.
‍

Research is self-serve and collaborative.

Rally’s user-friendly interface, proprietary study builder, and custom governance rules have enabled a safer, more self-serve approach at GitLab, democratizing research capabilities across the organization. Now, team members outside of the core research team can independently manage aspects of their own research projects. 

“With Rally, there’s added transparency to our outreach process and approach, making it easier to teach people who do research how to use the tool and become self-sufficient,” said Caitlin. It’s also enabled a wider range of stakeholders to contribute to and participate in research efforts. 

As the team adopted Rally, they saw benefits beyond just efficiency: “Rally has empowered teams like designers and product managers to handle some of their own research recruitment,” Karen explained. “Now, we can focus on high-impact, strategic research instead of getting bogged down in administrative tasks. We’re delivering insights faster and directing our time and effort towards research prioritization and business alignment.”

This shift hasn’t just empowered researcher, it’s also reduced the operational burden on Research Ops. “The ability to self-serve in Rally has been huge. It opens the door for the Research team to focus on foundational research, strategic research, and maybe even open the doors for other users, like designers, to help self-recruit on solution validation work,” Nick added. 

As more users become proficient with Rally, the Research team can concentrate on higher-level strategic initiatives, confident that their colleagues can independently manage the operational aspects of research.
‍

Research is transparent and compliant. 

Rally’s Governance features have enhanced the compliance of GitLab’s research efforts. By serving as the single source of truth for participant management, Rally enables the research team to confidently manage outreach and adhere to governance rules. “When I see ‘Free to Contact,’ I know Rally is telling me the user hasn’t been emailed in the last 30 days or whatever our rules are that we’ve set,” said Nick. “They meet all our criteria, and I’m free to contact them in a safe way.”

This has made it easier to track participant interactions and stay aligned with sampling and compliance requirements.

Additionally, Rally’s Salesforce integration provides a comprehensive view of user interactions across the organization, fostering stronger collaboration between UX Research and customer-facing teams. “We’re hoping to shed more light on who’s participating in research. Our Customer Success team is now more interested in getting involved with UX, which is exciting,” explained Caitlin. “The Salesforce integration and Rally’s Observer Rooms will be helpful in uncovering who is participating in research and close the loop that’s missing, hopefully leading to a better relationship between UX and CS.” 
‍

Research Operations are more scalable and impactful.

Rally has enabled GitLab’s Research Operations to scale more effectively. “With all these new efficiencies, we’re able to take on more requests,” said Caitlin. “We had 30 open requests at one point in June.” This scalability has been crucial as the organization grows, ensuring that Research remains a critical and impactful part of GitLab’s product development process.

As the Research team continues to leverage Rally, they anticipate even greater efficiency and a deeper integration of research insights across the organization, ultimately driving more informed decision-making and innovation.
‍

A few Rally features that ✨shine✨:

⚖️ Governance

“We had thoughts on how much the Governance setup in Rally would benefit us, but it wasn’t until we moved over our large-scale surveys that we realized how much it was helping. It’s shown us where we need to be more mindful, like with survey fatigue, and it’s helped with prioritization,” said Caitlin. 

Rally’s Governance features, including custom governance rules, contact statuses. And cool-down periods, enable both researchers and non-researchers to conduct secure and compliant research. The team can set opt-out links, manage participation limits, and implement custom rules, ensuring customers are protected from excessive outreach.

🔄 Seamless integration with Qualtrics

“I love that you can map Qualtrics to Rally properties. It was a game changer for us in participant management, and it showcased that Rally plays well with others,” said Caitlin.

The integration with Qualtrics allows participant data from screeners and surveys to be automatically mapped to Rally properties. This lets the team manage all their research data in one place, enabling more precise recruitment and panel creation through advanced filtering and segmentation derived from Qualtrics responses.

Rally = a need-to-have

Rally has become an indispensable tool for GitLab’s Research Operations. “If Rally was gone today, it would significantly impact our ability to operate at the speed and scale we need. There's just no going back once you've seen what's possible,’” said Will. He believes Rally supports modern businesses like GitLab, where speed is crucial. “Rally helps us automate the Research Operations process, which is essential for companies trying to move fast and build customer-centric products,” said Caitlin. “It makes everyone’s lives easier.”

Looking ahead, GitLab’s Research team is preparing to grow and evolve. A recent reorganization, led by a new VP of UX, has shifted the research strategy to prioritize measuring the impact of research efforts and aligning them more closely with business objectives. 

To support this, GitLab has expanded their use of Rally, purchasing licenses for their entire team of researchers to enable them to conduct independent research and leverage data more effectively. Key outcomes include:

• Collaboration between research, sales, and customer success will improve.
• Research will become an enabler for other teams to talk to GitLab customers so they can ingrain user feedback into strategic decisions.
• All researchers will be able to conduct their own studies, driving more impactful, data-driven research.

If you’re looking to scale your Research team or build out your Research program, Caitlin believes Rally is a need-to-have.