How do you build a research panel without getting overwhelmed by cheaters and AI bots just looking to snag participation incentives in exchange for useless data?

Amber Asaro, Manager, UX Design and Research Operations at Red Hat joined Rally on August 14th for an AMA and shared her secrets on how to screen out AI-assisted imposters from UX research.

Missed the event? Or want to revisit the key takeaways? Read the full recap below or watch the live discussion here. 

Download Amber's best practices for stopping and identifying participant fraud: 

Who Is Amber?

My name is Amber, and I'm a UX leader with over two decades of experience and a strong foundation in Organizational Psychology and UX Management. I've worked on UX projects for Microsoft, IBM, eClinicalworks, and Healow Health.

I came to Red Hat about four years ago as a UX researcher, and before that I had worked in electronic health records, also as a product manager and then as a usability specialist and researcher..

When I came to Red Hat, we had one researcher and one designer who were brave and raised their hands and said, 'I'll try doing the ops for the team'. And when I became a manager, one of the first things that I wanted to do was create more of an operations team that would support about 140 designers, architects, front-end developers, and researchers in the engineering org at Red Hat.

What does participant fraud look like (and why AI is making it worse)?

I think it's great to share these stories instead of being embarrassed or looking at it like a failing when somebody tricks you. It's really on them, not on you. 

I was doing a metastudy with another researcher, and we were doing deep interviews with 18 people, and we found out that we both interviewed the same person, and they knew that. And we shared that back with the team and it turned out another researcher doing a different study had just interviewed the same person, so they were triple dipping.

We've actually talked a lot about this as a research group, and it's sometimes really clear that in the free-write responses that AI was used. 

We rolled around the idea of using AI to detect AI, but in doing a little research, it seems like AI is actually not that great at detecting itself when you feed it back, and sometimes it creates either false positives or false negatives, so it's not reliable. And then a lot of people on our team we're distributed globally. So we have people in China and Israel, but our work language, our business language is English, and [overseas teams] use AI all the time to refine what they're saying before they submit it on things. 

So, it's actually pretty tricky, but one thing that we've done is just state outright in our screeners that we'd really prefer they not use AI and that we'd rather they have some errors that that's fine, that we're really just looking for authenticity and their unique take on things, so we, we prefer it not to be too polished.

Why does participant authenticity matter more than ever?

If we are engaging with human participants, it's because we're probably looking for a variety, we're looking for the spectrum of potential feelings or actions. We're looking for rich context, not the averaged-down response that you'll probably get from AI.

And we know that AI can have bias based on what's fed into it, and so it can actually amplify bias if we're reporting averages of averages from something aggregate like that in our research.

We're not just out to gather data. You can do that passively, and you can do that with computers, but we want individual human reactions to things which cannot be replicated. And most of you know that the most compelling evidence that you can sometimes put in front of stakeholders is those quotes or those visceral reactions on video or on audio. And so if you are presenting things that are false, then you're losing credibility.

And that also means that we try to bring stakeholders along for live sessions as much as possible. We have a whole calendar of live-stream sessions that we welcome people to attend.

What's the role of synthetic data in UX research (or, can AI be used for good)?

When I talk to my husband, he is a data guy. He's a more analysis-type guy that writes reports, and he said, what's the real difference? Because you can use synthetic data from AI and it could probably give you a wide range of responses.

But there really is a difference between carefully hydrated synthetic data that you're using to play around with. Maybe you've built it based on your knowledge of personas, and maybe you could use it for some things, but just fabricated data is polluted information. It's not trustworthy. 

It's important to sort of introduce the notion to your stakeholders of the need for triangulation, for seeing the same thing from as many angles as possible. So, for a quantitative perspective, synthetic data is interesting. But when you've decided that it's worth asking people, then it's because you need qualitative information or you want to uncover what you don't already know. So AI can't possibly know what isn't known yet and that discovery aspect of it is important too.

To what lengths will dishonest participants go to enter a study? 

I've encountered someone that actually created separate profiles. They submitted with a different name, a different location, a slightly different job title, and qualified for an interview study, and they won, they came on camera, and the second one they didn't. It just happened that I was very confident in being able to analyze in the video from the way that they spoke, and they did have certain verbal tics that a lot of times you're not aware that you're doing, little specific things that they would do that made and ways of phrasing things that made me really, really confident [they were a previous participant]. I really thought about this a lot because I didn't want to falsely accuse someone and say, 'I'm not, I'm not going to pay you.' But we had to actually remove that data from our study.

And so that meant we wasted an hour on each interview and then several hours coding them independently because we were doing that purposefully, coding separately to then compare and had to ditch both of those.

Then it turned out they were doing that with another researcher as well. 

So that one [incident] just led us to being very firm and prepared when someone said, 'oh, I can't be on camera.' Well, we'd already asked on the screener [whether] you have steady internet and that you're prepared to be on camera. So sorry if, if you're not available right now, we could reschedule, but we need that. And again, that visual can sometimes be the most compelling thing to present.  So, we just had to be more prepared to say, 'no, we're not going to proceed without you on cam.' 

How do you stop AI bots from overwhelming your sign-up process?

Then another one that was fun is one of our researchers had a neat way of getting a lot of responses to a survey. So instead of trying to pay each person 20 bucks for the survey, instead he took a large amount of budget for the survey and said that there would be a raffle at the end. That was supposed to get several hundred responses, and we had 25 responses like on a Tuesday night, and then when we came back on Wednesday morning, one of our ops people had posted it on a couple of Reddit threads and other social media, and we were thrilled on Wednesday morning, there were like 1,400 responses.

Then we started looking at them and realized that about 1,200 of them were from a bot. I think that the raffle was like 50 bucks. So someone had sat there and created something that would create different email addresses, different names for people. Fortunately, we could tell that they were all coming from the same IP address, so the bot didn't spoof the IP, but at least they did have different domains for the email addresses. There are a couple other things that made it very clear, also that the survey started to get completed in less than five seconds, and it was 20 questions, and that really spooked that researcher. That was about three years ago and they still are very gun shy about reviewing participants before accepting the data now.

We deliberately don't just have structured responses in our screeners. We always have at least one or two long-form, fill-it-in kinds of questions, and that's really telling, not only if a person is faking, but just if they're capable of putting their ideas in a succinct way and giving meaningful feedback.

What's the role of live video in participant recruitment?

You are on camera when we go through the interview. I think making that really clear initially just makes sense and then it's like no surprises and then if someone's like I can't go on camera, it's a little orange flag.

I also think that body language is also a really interesting component to having folks submit a video or having a camera turned on during an interview. I think you can also learn a lot about that.

How has AI changed the scale of participant cheating?

A member of research operations on my team came to me about a month ago regarding one product that was specifically designed, their tagline is 'we make it easy to cheat.' It's for tests, it's for live interviews, all sorts of things. It's glorifying cheating, and it's an AI tool that is invisible to others even when you're screen-sharing. So that's terrifying, that they can in real time, it's listening and then can generate a response for you that sounds credible.

So we're doing a fun little meta study where she's having people at Red Hat, without telling us, it's a blind study, they're going to choose one hobby or activity that they really know a lot about, and one that they don't know about. Then she's asking them the same questions, and they, ahead of time, have installed this terrifying thing, and we're going to, without knowing which one was which, we're going to go through and code what kinds of microexpressions or other indications, any body language, the way that their speech may be different, any type of clicking sounds that might be detected or eye movement that might indicate that they're reading, and we're going to code it up so that then we can compare once they do reveal to us which one was the new one. Again, are there any signs that you can watch for that will really give you a reliable clue? Because again, you don't want to be accusing people. That's the flip side of the coin. You don't want to be accusing people of doing something that they're not doing.

So that'll be fun. I think she's going to publish her findings to LinkedIn.

How can you spot an AI bot pretending to be a real user?

When you get those rich written responses in a screener, you could say something like, we were really interested in what you told us about that. And don't really restate what they said, and be like, can you tell us a little more about that? And that's not really that tricky to do to somebody that gave an honest answer. They'll usually know, oh yes, I think I told you about this book I had been reading, right? OK. And they can dig right into it. If they're constantly trying to game the system, they probably won't remember, or if AI generated it, they really won't know to follow up. So that's a great one.

A couple other ones are a lot of tools now offer a RECAPTCHA. It's annoying, but that little bot checker, some of them actually will generate a score for you on the responses on whether they think it's a bot, which is a nice yellow flag to make you just dig a little deeper on that.

IP addresses are not as useful now, but some people are lazy, and so if you are getting submissions from the same IP multiple times, it can be a clue.

Really generic looking emails can sometimes actually be a clue. Addresses that look like they were bunched, you'll just get a feel for it. Ones that start to look very similar, like they were created from some kind of pattern, we've seen that.

Definitely, we've seen a response that doesn't make any sense in certain fields, like a person's name, but in the job title field or something, they'll mess that one up sometimes.

And then, obviously, in moderated studies, they can be a little fishy. So if they're wearing a wig, or they're refusing to be on video, that's definitely a clue. And again, sharing, now we share in our codecs an image of the person that participated within a study, just in the PII folder, that helps too, like a quick clip, like to go with the name. And then obviously we purged that after the study closes, but it does help as a reference. Let me see what other people are saying though.

If all of the responses are so well formed and almost trying to show, like virtue-signal that they really know what they're talking about and they agree with you, even during the session, most of the time. I work with really deep IT professionals. If they don't have a really hot take on something, that's a red flag for me. If they're just, 'oh yes, we do it just like you say,' I'm like, come on, there, there's nothing that I'm asking you that doesn't make sense. It's a tough one, but you do definitely get a vibe for your target audience after a while.

Why is it so important to screen out every fake participant?

Yeah, and I feel like this kind of connects to the first question about what's the risk of having people falsify information in general. There are two sides to it. If they really are untrustworthy, then you lose credibility with the people that you're sharing the information with, and it's just not reliable intel.

But then the other side of it is if we're overly vigilant about this and maybe too strong about it, then we could be losing good faith with really sincere participants or even customers that are really trying to help if we wrongly accuse them. To keep strong in the game, I'm actually on a couple of research panels. I don't lie about what I do or anything, right? It's just, sometimes, you know, they have surveys about yard tools or something, and I garden a lot. So I don't get to participate a ton, but I have applied and participated for a few. And once, I got accused of actually lying on the screener.

And I knew that the screener was ambiguous when I took it. So I did my best, but probably didn't quite meet their criteria cause they'd phrased things in such a weird way. That made me angry. I'm still angry. But that really made me upset for like three months. I really had done my best, and then they didn't do a great job of creating the structure, and I got through and I even apologized and said, I don't think I'm the right person for this actually. So those are the two biggest risks.

How do you handle suspected bots?

If you're on the fence, when you get a screener back, instead of deciding that, 'oh, that's a bot,' what we've started doing now, sometimes we reach out to the person for a couple of follow-up questions or schedule a tech check call before scheduling them for the study. Or, somebody had that great idea of, just for the final round, submit a one minute video with this question. And those are ways to just test out your spidey senses rather than be too reactive.

Yeah, and so I'm not a person that endorses or is necessarily affiliated with any of these tools, but I've used a ton of them and also been a participant on some of them. So User Testing, the most recent knowledge that I have there is that after a session, the researcher can rate the participant and that rating, averaged, I think that it stays for the last maybe 15 sessions that the person did or something. So, if they get a certain number of really bad ratings, they'll actually be taken out of the panel.

If you can show that this person really tried to game the system, then you won't be paying that participant, and then the vendor will definitely have flagged that person and, and again, may take them out of the panel completely. So it holds them somewhat accountable, and not just for if they lied, but maybe if they really rushed through the study. If they really didn't read the instructions or skipped ahead or just answered questions 'yes / no' that clearly wanted a rich response. That's another option that some of those vendors offer that's helpful.

How do you find the right (human) participants?

There's the ideal situation, and then there's what to really expect in reality-land. So most vendors will tell you, and they do this, they will validate the email address. Some of them will get a LinkedIn or Facebook social media [link] and make sure that that lines up and that you're a real person there. They'll maintain a profile and keep that updated and, again, have ratings of former participation.

I think that that's great, sort of like insurance, but also I do really think that the researcher, by carefully crafting their screener, can mitigate a lot of the ability to game the system. In research, in both screeners and in our studies, we don't want to tip our hand too much. It's a tricky line because we want to be ethical, but we also don't want to tell them exactly what we're trying to learn.

Just because that can kick in, even for authentic participants, it can make them have that bias of wanting to please us and answer the way that they think. So, I guess it's both, but I wouldn't necessarily count on the venue as much because they're going for volume too, right?

They want quality, but they are not going to care if one in 1,000 people on their panel isn't that great. Whereas one out of five being lousy in your study can really ruin your study or hold you up.

How do you build a bulletproof participant screener?

So one of [Red Hat's screener tactics] is using a distractor. Either a low probability response or something that would be really impossible to be true. So for example, we have a pretty long list of what products do you use, and one of the products in there is Red Hat MacGuffin, which is not a real product.

And so, if this person is just trying to get through, they might select all of them, or if they select that thing, you know, either they're a liar or they're very inattentive, right? And probably not very reliable.

Another thing is to really ask for honesty up front. We want to hear from a variety of people with different expertise and technical backgrounds, so we need your honest answers. Asking someone ahead of time to agree to provide their honest answers, some studies have shown in some areas, not specifically in UX research, but in other areas like taxes and self-reporting, asking that at the beginning makes them more likely to say true things. If you ask at the very end, I attest that what I said was true, they've already lied now, right? If you ask it before, it can sometimes put that little guilt on them if they're not a professional cheater.

You can also randomize the screener questions, so that maybe the bot has a little bit of a harder time brute-forcing through it, because it's almost like cracking a code. 'Oh, it didn't like this answer, so now let me try this.' And if they're randomized, it's harder and again you might start seeing the responses being in the wrong fields at least for the free-typed answers.

Which again, always include at least one open-ended question in your screener, just to see the quality of what they're responding, how much mental energy are they willing to put into this and give you.

So, if you're screening participants for a longer survey or a moderated test, and you want to make sure they meet specific criteria for the really important stuff, instead of making it multiple-choice, ask them to write in their responses. And that does mean, yes, you're going to have to manually review that, yeah, but it's much less likely that they're going to magically choose that product that they use that you want to hear about, right? Uh.

That'll do two things too. It makes it harder to come up with the 'right' answer, and you can see how much flavor they give in their response.

Another trick, and I used this when I was working with Microsoft. There were only like three or four big browsers at the time. So sometimes I would say, 'which of these browsers do you use?' and list three that I really didn't want to talk to and then have 'Other,' and they could write in a response. And so, if they said 'Other' and put that browser in there, that indicated to me that they probably did use that thing, right? Because when you're looking at a screen, you're like, 'oh, I don't use Google or Mozilla or Duck-Duck-Go; they probably don't want to talk to me, but I'll put in that I do use Bing.'

Instead of asking somebody, 'do you do any of these three things for part of your job?' you can ask, 'how frequently do you do these things?' with a little matrix. We'll find that, if somebody's really trying to get through, suddenly they'll do all these tasks every day that you'd have to have 50 hours in your day, probably, to do. It just doesn't add up.

Finally, check for the duration that it took for this to be completed. I'm probably giving away the secret. Somebody's on here and going to use this, but a lot of times AI isn't smart enough to go slowly. If it should take five to seven minutes and it's done in 20 seconds, that's a hot tip for you.

This is so tricky because we've actually had people come back to us a month later and say, 'you didn't pay me for the study,' and it was the screener. It was just so long that they thought they were doing the study. You don't need a screener to be more than five to seven minutes. That's still pretty steep if you're going to be sitting and interviewing with a person. They don't want to invest that kind of time, or they're not going to apply again with you anyway. 

If you're using a vendor, same thing, they'll probably give up a certain amount of way through. Qualtrics and some other tools will tell you about how long something will take and give you a score on the complexity.

It just depends. I'm comfortable with having [the screener] involve a little more work because that shows me that it's a person that will probably take the study and show up.

One of the biggest things that I did not mention is consider not ending [the screener]. You can have the logic set up or the minute they answer the wrong thing, you say, 'OK, thanks.'

That is essentially telling a bot, 'wrong answer.' If you just let it get all the way through anyway, which means you're wasting humans' time, [the bots] don't know which ones were right or wrong. It's not giving them that feedback.

One more benefit of [not ending the screener] is you still get that information about those people, and we always ask, can we get your email address and contact you for future studies? So, all that information is still very helpful for the profile, and then we can reach out to them for something that is more suited.

How do you effectively incentivize honest, human participants?

I'll call out my colleague Flo. She actually did her master's thesis on what motivates people to participate in studies. So I actually did ping her and definitely considered what she thought about this.

A lot of times if you're trying to talk to an IT decision-maker. They probably make $100 an hour, so they're not necessarily that motivated by that $20 incentive fee.

We found that sometimes it's more effective to say that once we've reached the quota for our study, we're going to donate $2,000 to one of five charities and you can vote for it at the very end of the survey. So it motivates them to complete it, and it motivates them, hey, share this with your colleagues. It appeals to their altruism and motivates them to get all the way to the bottom.

Then it's important to get back with them and say, 'hey, thank you so much, and we donated this to this charity, it's the one that won.'

Sometimes for our customers, when we really want to talk to people that are a little brand loyal, we can offer Red Hat swag or some kind of status, and that appeals to people a lot more. Again, only for certain types of studies. But that is a great opportunity to not be putting something out there that people want to cheat to get.

Another thing is just closing the loop and coming back and telling people later, hey, this was the overall what we got from your feedback and what we're going to do with it. People are a lot of times motivated because they want to make a difference. They either care about the field or the product, and so appealing to that instead of offering money when possible is a really great idea. In healthcare, where I was before, I could not offer any type of incentive or exchange of value at all, so I had to lean very heavily on that.

Do you hate things about our software? Well, this is your chance, so consider those as alternate ways to incentivize, too.

Lotteries can incentivize fraudsters sometimes. So again, this was more for a charitable donation that works better than winning a prize. 

How much should you invest in participant screening?

First, you should really reflect on [your screening overhead] if you haven't. Just look at your process and build [these tips] into the templates for your screeners. The way you phrase questions, some mitigating techniques, should really just become part of business as usual, so it's not something isolated that you do for a specific study necessarily. That's the ideal and that's what our researchers tend to do at this point.

At Red Hat, I actually put little Easter eggs or more colloquial language in [to the screeners]. Like, 'ooh, you made it to the halfway point, stick with it.' Just little things that a human being will find entertaining.

Also telling them, when asking a question, 'go ahead and write a novel; a human being will read every word.' They're wondering the same thing, 'is anybody going to actually look at this and read this, or is it to get screened by AI?' So telling them, we're going to eat up every word of what you put here. Telling them what this is going to be used. You can't be too specific, but it's going to change the world in this way.

You can inject some fun, or at least some excitement into it.

Connect with Amber

Follow her on LinkedIn and say hello!

Listen to Amber share her insights on building high-impact, honest, human-only research panels and survey groups.

Thank you, Amber!

An immense and sincere thank-you to Amber for sharing how she and Red Hat are defining and demonstrating UX Research best practices, especially in the age of AI fakery. While the ways artificial intelligence can skew or even wreck research can be scary, Amber's insight gave us the tools to adapt and the confidence that good data - and good participants - can still be identified, nurtured, and engaged. We're grateful for the time and expertise she shared.

If you'd like to watch the full AMA, follow this link.
‍

Download Amber's best practices for stopping and identifying participant fraud: 

‍