Demystifying User Research in B2B
To continue our exploration of the Rise and Rise of the UXR, we are diving deep into the world of B2B. We interviewed a handful of UXRs in the B2B space and learned a lot. To kick off our journey, we want to share some of our learnings. And if you stick with us, you can join a few of our B2B UXR rockstars as we dive deeper into topics like:
- Recruiting niche users
- Measuring impact in B2B
- Joining a B2B research team
- Finding unbiased voices in research
Introducing our B2B UXR Heroes:
- Shya Castillo, UX Researcher at Culture Amp, an employee experience platform.
- Andrea Willimetz, Senior UX Researcher at MongoDB, a developer data platform.
- Johnny Ross, UX Researcher at Pendo, a product experience software.
- Matt Morrison, UX Research Manager at Braze, a customer engagement platform.
- Taylor Jennings, Senior UX Researcher at Chili Piper, a meeting lifecycle automation platform.
- Bernardo Vailati, Senior UX Researcher at Pipefy, a workflow management software.
What do B2B research teams look like?
No two research teams are the same. And how a team is set up and where it sits within the wider organization is different depending on the size of the team, the industry, and the needs of the company. For example, the research team at Braze, where Matt Morrison is the UX Research Manager, sits within the product org in the UX department, which includes design, research, and writing.
At MongoDB, researchers are embedded into smaller groups and have “become experts in those areas so we’re able to contribute more, engage stakeholders, and encourage them to participate in research with us,” said Andrea Willimetz, Senior UX Researcher at MongoDB. Sometimes researchers are embedded onto teams where they have previous experience, like UX Researcher Johnny Ross, who had previously worked on growth initiatives and was placed on the growth team at Pendo.
Not every research team takes the embedded approach; the researchers at Culture Amp “support specific product areas” instead of embedding themselves on specific product teams, said Shya Castillo, UX Researcher at Culture Amp.
Before Taylor Jennings joined Chili Piper as a Senior UX Researcher and their first research hire, the product team was handling research. “They were doing it ad hoc and not super methodically,” she said. “They realized research is a function we need and we’re at the point where product and design really can’t do it anymore.” Now Taylor has another researcher on her team to help handle Chili Piper’s research efforts.
Some B2B research teams are quite robust and include Research Ops and Research Leadership, while others, like Matt’s team at Braze are still working to expand and add those roles to their team. Ideally, Matt’s ultimate goal is for research to “have more leadership and a seat at the table alongside product and design leadership.”
How do B2B organizations think about research?
Everyone at MongoDB understands the value of UX research — both for large foundational projects as well as small usability ones. While they understand the value, “oftentimes there is a lot of emphasis on analytics.” But Andrea has seen a shift to “triangulating to uncover the truth behind questions” using both quantitative and qualitative insights.
For Culture Amp, “there’s an appetite for research and an understanding of why research is valuable,” said Shya. “You’re not constantly having to advocate for why research should be done.”
Product and Design teams at Chili Piper when Taylor joined were curious and supportive of research but limited in their methods and understanding, only doing moderated usability testing. “I think our product and design team really thought that was the only way to do research.” So Taylor worked to open a new world of research by educating her teammates at Chili Piper on different methods and tools.
A culture with an appetite for research is a great start for research teams. “But if you want to transform the company culture to be data-informed, making evidence-based decisions, you really need to let people experience what it’s like to talk to the customers themselves,” said Matt.
What is the state of democratization in B2B?
“It’s a necessity, right?” said Johnny when we asked about democratization at Pendo. “Especially as your company is scaling, it’s difficult for researchers to address smaller initiatives.” Across all industries, Johnny believes “the activities we are democratizing are relatively the same. Usability testing, short surveys, and some user interviews tend to get delegated to the product teams after training with a researcher.”
For Senior UX Researcher Bernardo Vailati and his team at Pipefy, “we wanted to empower people to do their own research and we’re now at the point where we can’t guide people as much because we don’t have the time and capacity.”
Democratizing research is important, but for Bernardo, it’s even more important to do it well. “We need to have systems, processes, and best practices in place.” The demand and desire for research is a powerful and exciting thing in a B2B organization, but Bernardo and his team want to ensure it’s being done in a responsible and ethical way with as little bias as possible.
For Taylor, allowing people access to information and granting the ability to self-serve is an important part of democratization at Chili Piper and for research in general. “This means having a research repository and making sure the tags make sense. It should be up to date and also a place for people to access study findings” she said. “If people want to know what’s going on with research, they can follow along at their own pace.”
Another essential decision to make when democratizing research is deciding who owns which pieces, something Shya and her team have spent time thinking about. This involves determining, for example, if the research team should own recruitment or merely oversee it.
Some, like Matt, are a little skeptical and cautious when it comes to democratizing research. “I think with proper education and enablement, democratization can work. But if everyone is just doing it free reign, then research isn’t going to grow in the way it should and you may get some fossilized mistakes.”
What does success/impact look like for research in B2B orgs?
“Impact is kind of a nebulous term.” That’s what Matt told us when we asked him about impact. And he’s right. Both impact and success can feel a little vague and are also usually specific to your organization, team, and role.
To wade through the haze of impact, Matt follows Verizon’s Director of UX Research, Victoria Sosik’s, model on impact, which breaks impact down into three categories: product impact, cultural impact, and internal impact.
For Shya, “first and foremost, success is the relationships.” When determining her and her team’s success, Shya reflects on questions like:
Specifically, Shya considers being seen as an important indicator of success, especially when it’s in relation to the “strategy and where the product is going to be in 2-3 years.”
What is one of the biggest challenges you face in B2B?
With hardly any hesitation, nearly every person we spoke to immediately said, “recruiting.” So, what’s especially difficult with recruiting in B2B?
In B2B, recruitment is extremely specific.
Bernardo explained that the criteria for contacting users in B2B are more qualitative and not as easy to find in a data set or through product usage metrics. For example, data won’t be able to explain how a user feels about certain aspects of the product.
Specificity also comes into play when you work at a company that has a very finite and distinct user. That’s a challenge for researchers like Andrea. “It’s difficult to figure out the right initiatives or incentives and determine the specific groups we want to talk to within our segment of users.”
MongoDB’s primary users are developers who, “especially in a business environment, are really busy.” Andrea’s preferred one-on-one format is an hour-long interview, which is often difficult to fit into a developer’s already packed schedule.
Despite these challenges, Bernardo feels there are great advantages to recruitment in B2B. “We have way more data that is structured, so it’s easier to find people despite the relationships they have with the products being more complicated and complex.”
Another thing he does to combat the specificity of B2B recruitment is to lean on Pipefy’s internal team like sales, customer success, and support. Often Bernardo will ask, “Hey, have you seen this person? Do you know people with this profile?” This has helped point Bernardo and other researchers to the best users to speak to.
What does cross-collaboration look like in B2B?
How specifically does Bernardo utilize the internal team at Pipefy? Each member of his research team has a strong relationship with members of other teams. For example, Bernardo has strong relationships with members of the data science team while another researcher with members of the CS team.
Using those relationships, Bernardo has strengthened his research and often reaches out internally before starting any research to see if other teams have any background on the problems he is researching or can connect him to ideal customers.
Like Bernardo and his team, Shya utilizes members across multiple internal teams like design, content strategy. “The type of research we are doing will also influence the stakeholders we are working the closest with,” she said. Often, Shya and her team work closely with PMs. “They’ll review interview guides, and help with recruitment since they have strong relationships with the customer side of the organization.”
PMs are often staples when it comes to research cross-collaboration. Another role that researchers will turn to is customer success. “They are our biggest touch points,” said Matt. “We go through them a lot because they want visibility into how Research is engaging with their customers.”
And in smaller B2B companies, like Chili Piper, when it comes to research, it’s “all hands on deck” and there are very strong relationships across the entire organization. “People understand the inherent value of research throughout the organization, which is more than I can ask for,” Taylor said.
How is B2B different from B2C?
Aside from surface-level differences between the two, we heard a variety of deeper differences. One of the biggest differences was regarding the product and how complex B2B software can be for both researchers and the users they are talking to.
Bernardo is no stranger to software, having always worked with web apps and in eCommerce, but when he joined Pipefy, which had a “full-blown seven-feature-deep software” he found himself unprepared. Pipefy’s product and its relationship to users proved challenging and complex to test.
The relationship a B2B user has with a product is quite different from the relationship a consumer will have with a product. “Some of the feedback you get from B2C users might not be useful; you may talk to users and they’re not quite sure what you’re talking about or why,” said Matt.
Braze’s users are developers and marketers who are using the software all day, every day. “They have a lot of feedback and they’re more willing to give that feedback to us. It’s not a platform that they occasionally shop on,” he said. Before joining Braze, Matt worked at Etsy where he encountered users whose activity in the product was irregular and not as essential to their day-to-day activities as Braze is for marketers.
Users eager to give feedback can be a blessing and a curse, as Shya explained. “We have people who are really vocal because they hate something in the product or because they are fans. We have to be really careful that we aren’t biasing ourselves with people who are really eager to give feedback.”
One thing Shya and her team do to avoid bias is to consider all perspectives. “We try to reach folks that maybe aren’t as vocal or enthusiastic about the product.”
Johnny believes one way to consider all perspectives is to branch out and tap into non-customers or find “those fringe customers that are more early majority as opposed to early adopters.”
What was unexpected in B2B?
“Everyone in the company is talking to our end users all the time,” Matt told us. Matt was surprised by the large commercial arm of B2B businesses and the dozens, even hundreds, of contact points users have with the company. “All the ways they are talking [with users] are very different. You really have to think about what research is trying to achieve vs. sales.”
Andrea was surprised by how willing and supportive the user community is at MongoDB. “They are very willing to continue to improve the product and give their input in whatever way is helpful.”
Taylor’s experience at Chili Piper was similar, “People are just willing to try to help make a thing they use every day better and are happy to break down the intricacies of what they do and how they use things,” she said. “It kind of renewed my faith in humanity just a bit.”
Many B2B companies have very technical products. “At least some technical knowledge is important,” Matt said. “What is an API? What is an SDK? What is a webhook? How does that make sense within my company’s ecosystem?”
But don’t worry, Matt reassured us that this knowledge comes with a lot of practice and no one understands everything immediately.
Advice for those new to B2B
If you’re a researcher in B2B, we’ve compiled some advice for how to excel in your role.
Be a storyteller.
“The end of your work results in the share-outs you create,” is what Andrea told us. When she first began sharing her research at MongoDB, she would mostly just include insights, observations, and recommendations. “That’s great,” she said. “But what’s the bigger story you’re trying to tell with your research? How can you tie it into the larger initiatives or company goals?"
Keep it small, keep it fast.
B2B is complex. “If you’re going to try to do a research study that’s fully comprehensive, it could be months.” At Pendo, Johnny’s learned to narrow down the question “and just pump it out.” By honing in and making sure research is building on other research, you can get multiple studies done in a month. “There’s a lot of paralysis when it comes to B2B because it’s so large, ultimately just pick the smallest thing and run.”
Be creative and think ahead.
As we mentioned above, recruitment is a nightmare. Matt’s advice was to be creative with your recruitment strategies, something he wishes he had done sooner in his B2B research career. Another strategy he proposed for helping with difficult recruiting is to put together a continuous research program “where you’re talking to the same small panel of users repeatedly or have different people every time — ultimately it should just be a program that is set in motion.”
Utilize the research that’s already been done.
“It’s kind of madness to start all research from scratch in B2B,” Bernardo told us. Many B2B companies already have a vast wealth of data and research, which Bernardo said can be extremely helpful for recruiting users, interpreting your own research, and making your research more impactful.
Focus on the relationships.
For Shya, having really strong relationships with stakeholders across the organization is key. “Don’t just partner with those who are naturally closest to research, like design,” she said. Relationship building involves taking the time to get to know people, Shya explained. “Learn how they are thinking and their perspectives, and what inputs they have that are driving their decisions.”
Join the research community.
The research community is tight-knit and eager to help each other. “People are just idea-sharing and there is no competition,” Taylor said. Staying plugged into that community is a great way to learn more and connect with people who “are already doing the thing you are or the thing you are wanting to do.”
Transfer into research from a CS role.
Johnny explained that it can be difficult for someone who is a first-time researcher (welcome to the club 😎) to get into B2B because often companies want people with previous experience. “If you are able to, try to maybe take an analyst role or a customer success advocate role, and then once you have a better understanding of the customers, you can transfer into the research role.”
Ultimately, though B2B may not be as sexy as B2C, Matt believes “it’s a great environment to work on some really fun, challenging problems.”