Subscribe to our mailing list

* indicates required

Market research, survey analysis and productivity.Subscribe Now

Overcoming the Challenges that Come with Conducting Online Surveys

Conducting Online SurveysSetting up your online survey and collecting data seems easy enough, right? This can be true with an increasing amount of online survey tools at your disposal. Software like SurveyMonkey and DataCracker have taken the pain out of data collection and analysis. Although it is simple enough to create some survey questions and ask people to complete a survey there are also some hidden challenges that need to be overcome.

How do you attract meaningful and honest responses? Have you considered the way that your questions will be interpreted? Have you thought about the problem of predicting behaviors? These are just a few of the challenges that you will need to consider if you intend to uncover real insights that you can use.

In an earlier post I covered the secrets of getting actual responses to your survey questions. Below are some tips on how to make sure that the data you are collecting is useful and that will create some real and actionable insights for you to work with.

How do you incentivize meaningful and honest responses?

How do you incentivize meaningful and honest responses?The first question you need to ask when attempting to incentivize your survey responses is “Why will this person take this survey?” or “What’s in it for them?”. One of your first thoughts may be “Can I offer them financial reward for the survey?”. The danger in purely monetary reward might lead to rushed and unthoughtful responses motivated purely by some extra cash. Maybe you don’t need to throw money at the situation to get the best quality data.

So what are your options?

  • Consider a donation based incentive feature for your survey. By appealing to someone’s sense of charity you will be more likely to ensure only genuinely interested respondents take part.
  • Substitute hard cash with something like an Amazon gift card is another tactic that could attract better responses.
  • You may also try and appeal to the respondent’s sense of purpose. Make the incentive about their contribution to shaping your product or service. Our natural inclination is to reward tasks with money. Appealing to people’s more intrinsic motivations may be a good way to ensure you get more honest and meaningful survey responses.

How can you make your survey questions bulletproof from ambiguity?

Start with the end in mind. Know that once you have collected your data you need to be able to discern what it means. A good way of checking for ambiguity in questions is to use think aloud interviews. This is where real respondents are asked to answer the question but are required to verbalize all their thoughts while answering the question. Test your questions on a family member or co-worker and look for any evidence of ambiguity that may hinder your insights.

The following question is a simple example that illustrates an ambiguous survey question:

Question: How often did you seek treatment for your back pain last year?

Response: 33

If our goal with this question was specifically to understand how often the respondent visited a chiropractor last year then we have created an ambiguous question. The respondent may have visited a chiropractor 17 times, a physiotherapist 10 times and a massage therapist 6 times. We don’t know the precise information that we set out to find.

How do you encourage accurate prediction of behaviors?

Another problem you may encounter with your survey questions is when asking people to predict future behaviors. People don’t tend to predict long term behaviors accurately. Circumstances change and therefore our original predictions do not match up with reality. One solution to this problem is to survey people on something that is as current and specific as possible. This will protect against having your respondents predicting inaccurate behaviors.

You might set out to find out whether your respondents would buy your new product. “Can you see yourself buying a case for your iPad that you wear over your shoulder?” Most people will find it difficult to answer this question accurately as they are unlikely to have thought about the need for this product. They will be making a prediction on future behavior based on their current behavior. By adding a narrative for your intended target market you are drawing an response that is based on a clearer and current behavior. For example: “When riding your bicycle to and from work would you consider buying an iPad case that you wear over your shoulder?”   

When drafting your questions for your next online survey be sure to give plenty of thought to the types of questions you are asking and the precise working of those questions. Test your survey out verbally on family or co-workers and make sure you haven’t missed any obvious problems.

Good luck! Let us know in the comments if you’ve had any experiences with overcoming these types of challenges with your online surveys.


The article was originally posted on
Image courtesy of stockimages and Jeroen van Oostrom /

About Luke Ryan

Luke Ryan does marketing for DataCracker. He writes on the blog and works on community and coverage. He loves to answer your questions about all things DataCracker.