Writing survey questions may seem an easy task but it really is an important phase of your research and you can frequently occur in survey errors. How you write questions may determine your data collection success or failure.
That’s why we want to list the 5 most common survey errors: note them down and stay away from them!
Common survey errors
1. Response bias
As ambiguous words don’t have a univocal interpretation, they confuse the respondent and corrupt survey data. Let’s make an example:
“Where do you like to shop”?
In this case, the faulty word is “like”: it’s too generic. With “like” do you mean shops where the staff is nicer? Where prices are more convenient? Or where the atmosphere is best?
Why you have to avoid errors and biases:
- Each respondent will interpret the question differently so answers cannot be compared
- The respondent will not answer rationally but he will pick an answer he thinks looks more coherent to his situation
2. Double negations, passive, too complex questions
Keep it simple – that’s what communication gurus say. Any complex structure can negatively influence your survey research.
Example: “In which store you prefer not to shop?”
This same question can be phrased as:
“Which store do you shop in?”
In this second query we:
- Avoided negation;
- Removed “to prefer”;
- Shortened the phrase from 8 to 6 words
Our tips: review all questions avoiding negations, passive verbs, long wording, and complex verbs. The respondent will focus on the question more and you’ll increase your response rate.
3. Double questions
When you write a question it’s common to ask for multiple things.
Example: “In next elections, you will vote for the candidate who will increase public expenses on welfare and instruction?”
In this case, if you vote for a candidate who supports instruction (and not welfare), you will not be able to answer and you will be forced to give an answer that is not true.
So, double questions increase the possibility to get ambiguous answers: if possible, avoid them.
4. Absolute questions
With the word absolute, we indicate all those questions that include a “never”, “always”, “every”… This type of question often leads to statistically useless answers.
Example: “Do you always have breakfast? (Yes/No).
In the example, we force the respondent to give a generic answer (all those who skip breakfast once in a while will be forced to answer “no”) and we give to the question too many possible interpretations (is brunch considered breakfast? What if I just have coffee?).
5. Open-end questions? Yes, but…
Open-end questions collect more detailed data and for this reason, we need to use them. But use them carefully!
Let’s put ourselves in the respondent’s shoes: how would you feel when you notice that the majority of questions include empty spaces to fill-in?
Two possible reactions:
- he joins the survey but as he proceeds into it he’ll give more and more summary answers
- he will drop the survey
To sum up, we suggest to carefully distribute your open-end questions only when you need a detailed answer.
