Every survey goes through a set of behind-the-scenes checks the moment a respondent hits submit. A valid survey passed those checks. An invalid survey didn't, meaning something about how the survey was taken makes the results unreliable. That could mean a bot took it, the respondent wasn't paying attention, they were trying to manipulate the outcome, or they answered in a pattern that doesn't hold up statistically.
Not every flag makes a survey invalid. Some are advisory. Here's what the labels mean when you see them.
Signs of manipulation
"This respondent appears to be manipulating the survey"
If a respondent looks like they were trying to figure out how the survey works or get around it, we flag it. That includes things like jumping between browser tabs a lot, taking screenshots, or opening developer tools. When enough of that behavior shows up on a single response, the survey is invalid.
Social desirability responding (SDR)
SDR is when a respondent answers in a way that makes them look overly good. Endorsing every positive adjective and rejecting every negative one, past the point of realism. Saying Charismatic is Like Me and Nervous is Not Like Me is fine individually, but a whole survey of that pattern isn't credible.
Two flags to know:
"This respondent is strongly managing their impression"
The respondent went past the allowed threshold of favorable-direction answers. This one invalidates the survey.
"This respondent may be managing their impression"
The respondent landed close to the threshold but didn't cross it. The survey is still valid, but flagged. These respondents are permitted to retake, and it's recommended they do.
Timing
"The respondent took too long to complete the survey"
Each survey has a time allowance. Personality, for example, has a 30-minute timer. If a respondent takes longer than that, we can't be sure they were still engaged with the questions. Similar to SDR, the survey is still valid, but retake is recommended.
Attention checks
"This respondent failed attention checks"
The survey mixes in two kinds of attention checks. The obvious ones are direct instructions like "Select Like Me," where the respondent literally has to follow the instruction. The softer ones look for contradictions between opposite adjectives. If a respondent misses too many of either kind, the survey is invalid.
Signs of automation
"This respondent appears robotic or responded unrealistically"
The system also watches things like typing rhythm and mouse movement. When those look robotic or too perfect, in patterns a real person's hand wouldn't produce, the survey is invalid. This usually catches bots.