Subscribe to our mailing list

* indicates required

Market research, survey analysis and productivity.Subscribe Now

Diagnosing Customer Experience Problems with Market Research (and Doing It Economically)

Diagnosing customer experience problems with market researchYou have a suspicion that something is wrong.  Sales are going in the wrong direction. You are hearing the odd complaint.  The vibe is generally bad.  But how do you work out what the real problem is?  And indeed, how do you work out if you really do have a problem?

There are lots of tools available for detecting if we may have a problem.  We can give customers the ability to rate our performance on our sites.  They can rate us on evaluation sites.  We can compute our repeat rates for sales and our product return rates.  We can monitor customer complaints and social media.

However, once we get the smell of smoke we have a problem.  The old adage is ‘where there is smoke there is fire’.  But, it is also true that we should be all things to all people. The only solution is to get objective data.  Market research is usually the best tool for this problem, as it allows us to ask precisely what we want to know.

Fortunately, there is a standard process that consultants use for diagnosing customer experience problems.  It is a three step process.

Step 1: Identifying all the possible problems

We start by coming up with a list of all of the possible causes of customer experience problems.  This list usually includes things like price, speed of resolving problems, ease of use of websites, friendliness of staff and so on.  Commonsense will get you most of the way there.  A few chats with disgruntled customers can fully flesh-out the list. If you do not have the time, a Word Cloud [future blog post] is often a quick and easy way forward.

Step 2: A customer experience survey

Surveys are the bedrock of understanding customer experience as they allow us to move from hunches to data.  The trick is to get a random sample of your current customers and then ask them questions that permit you to understand what is wrong. Check out for some tips and instructions for doing your own surveys.

There are hundreds of different platforms for conducting surveys and many let you create basic surveys for free.  I have put together an overview of some of the main tools.

Customer experience surveys tend to follow the same structure.  They start by asking a series of questions that help to profile the customers.  In a business-to-business market it is common to ask about industry and number of employees.  In a consumer market it is usually necessary to measure things like age, gender and marital status.

Then, the questionnaire needs to ask about where, what, why, when and how they buy. For example, if you have have a online retail site you would ask people things like which categories they purchased, how often they purchased and how they paid.

A single overall performance metric needs to be used.  Most commonly this is a question about satisfaction, such as:

Survey question - Microsoft Office

Most big corporates ask people how likely they are to recommend their product, on a scale from 0 Not at all Likely to 10 Extremely likely.  Alternatives include asking people to rate their likelihood of using again, or, whether the product was worth what was paid. Most metrics give the same basic answer, so there is no need to get too caught up in this.

Then the questionnaire moves on to collect performance scores and importance scores. Performance scores measure the actual performance (e.g., “How satisfied are you with the following aspects of PRODUCT NAME”). Example:

Survey question - iPhone

Importance scores are collected with questions like “How important are the following things to you when choosing a PRODUCT DESCRIPTION”, and then presenting the list of drivers and a getting people to give ratings (e.g., out of 10). Some consultants do not ask the importance questions and instead use a statistical technique called regression to derive importance scores by looking at how the performance scores relate to the overall level of satisfaction.

It is usually a good idea to finish with a general catch-all question in case you have failed to ask about something important.  For example: “Is there anything else you would like to say about YOUR PRODUCT?”

Step 3: Analyzing the data

Analysis starts with the overall performance metric.  Scores below 7 indicate a problem. Scores of 7 and 8 mean you are doing an OK job and have room for improvement.  If most of your customers give scores of 9 or 10 then you are doing a great job and can regard your analysis as being finished.

Analysis then moves on to understanding the differences between customers that give high versus low ratings.  Do men give lower ratings than women?  Does performance differ according to which bits of your business the customers interact with?

The easiest way to do this is to compare averages in the overall performance metric between different sub-groups of customers.  A fancier approach is to use a predictive modeling [future blog post].

If you are great with numbers you can do these types of things in Excel yourself. Many of the data collection programs also allow you to different types of analysis.  However, if you don’t have a great feel for numbers it is worthwhile using an analysis program designed for analyzing surveys, be it my own product, DataCracker, or one of its competitors, such as MarketSight or SPSS.

All these products have free trials. The key reason to use a specialist product is that they automatically tests for statistical testing.  While statistical testing can be a confusing thing to understand, it is the easiest way of working out which results are trivial and unimportant, and this is a key thing to find out, as without knowing this you will end up wasting a lot of time fixing things that are not broken.

The final step is to put together performance-importance charts, such as the one shown below.  This one, done for a bank, shows that the bank’s key problem is in Price, which is in the top-left corner (i.e., very important but poor performance).  The bank has perhaps over-invested in Online Banking (Higher performance but less important).  And, while the performance is weak on Branch, this is very unimportant, so this is not a cause of customer experience problems.


A last word of caution. There are enough freemium products out there that you can diagnose customer experience problems for free. However, the diagnosis will only be as good as the effort that is put into ensuring that you talk to the right customers and ask about the right things, so make sure that the person who ends up doing this is the kind of person who likes to get into the detail.

If you find this post useful, share it.


Image courtesy of Master isolated images at