While the old English saying is true about how asking too many questions can turn ugly in certain areas of life, being inquisitive about your audience is key to an effective customer experience strategy.
Still, it’s important to be equally attentive to trends and changes in audience behavior, as it is to determine a cohesive survey design approach.
So, how do you, as a marketer or user researcher, find the golden mean between structure and constant adjustment?
I’m going to show you how understanding the fundamentals of survey methodology can boost the quality and relevance of your feedback collection strategy.
You’ll learn how to set your survey efforts up for success with the right tools, and types of insights you’re looking to derive.
Regardless of whether you’re new to survey design – or already have some experience, but never got around to catching up on the theoretical basics – you’ve come to the right place.
All set? Let’s begin!
What You Need to Know About Survey Methodology
If you search for the term “survey methodology” online, you’ll see it’s used interchangeably to describe all sorts of survey tools, questionnaire construction methods, or feedback collection hacks.
In fact, they all somewhat fall into the definition.
According to UCLA Labor Center, “survey methodology is the study of survey methods and the sources of error in surveys”.
Errors are all the factors that deviate your survey efforts from the desired outcome. Survey methodology studies aim at minimizing their occurrence in the future.
Now, depending on your previous experience with feedback collection, the word “survey” may bring various pictures to mind.
For some, it’s that evening call from the national opinion center prior to presidential elections. Others are more likely to think of the occasional website pop-up on their favorite online store, or live polls carried out in the street.
These are all examples of various survey instruments. But before we dive in, let’s think about the very core of all feedback collection:
Why do you want to survey your audience?
Are you planning to collect feedback as a reactive measure to a single event, or do you want to analyze results for the same question across various points in time?
Are you looking for structured responses that can be quickly analyzed with a data analysis tool, or do you want descriptive answers to open-ended questions?
All these questions, among others, need to be taken into account if you want to optimize your survey design efforts.
So, let’s take a look at the pros and cons of various survey tools, and consider how some of the most popular customer satisfaction metrics can serve your goals.
At the highest level, surveys can be divided into two groups: questionnaires and interviews.
In questionnaires, it’s the participant, whereas, during interviews, it’s the researcher who notes down not necessarily answers, per se, but key takeaways.
Now, at first glance, you might contemplate drawing a line between the two – especially if you’ve come to this post with a plan to collect feedback through questionnaires only.
Potentially, interviews may seem like too much of an elaborate research method for your current needs. And, chances are, your gut feeling is just right.
Still, the thing is, this might change quickly.
As you derive more and more information about your respondents, there may come a day longer conversations will make a lot of sense – and a world of a difference for your product development plan. And, when the time is right, you’ll know exactly what tool to employ to expand your feedback collection efforts.
Another great thing?
Interviews and questionnaires are pretty much a marriage made in heaven. While they’re doing quite well solo, combined, they’re a whole other dimension of powerful feedback. One method can accelerate the efforts of the other – interviews can complement what you’ve already learned from questionnaires, and questionnaires can pre-qualify the right people for interviews. Pretty great, right?
So, let’s take a look at the ways you can ace the feedback collection game.
In a digitalized world like ours, no one needs to be convinced of the incredible potential of website feedback.
These surveys take on various forms – from large pop-ups in the middle of the screen, to discrete widgets on the edge of the browser window.
Here’s a general breakdown of what you can expect from website questionnaires:
With the right tool, you can target an audience as broad or narrow as you like
The fastest way to collect crucial feedback from your website crowd
With Survicate, you can target relevant respondents right from the beginning, as you can easily import data you already have on your audience from external marketing and CRM tools.
Depending on your preference, you can also choose to run website questionnaires through your favorite customer communication tools. This approach, loved by many marketers, allows you to use the same tool for all audience communication, as well as cross-analyze and export survey responses within seconds.
Similar to website questionnaires, mobile app surveys are displayed to users who interact with the product/service.
Ideal for companies that have mobile apps and want to know how users find their way around/what they think about the service
You can reach users “on the go” – ideal for single-question surveys that are quick to answer
You can target users contextually and ask highly relevant questions.
The threat of being quickly discarded, if too many steps or open-ended answers are required (i.e. inconvenience of typing on a small screen)
If a questionnaire covers the entire screen or appears too often, there’s the risk of coming across as obtrusive.
Let’s carry on to the last online questionnaire on the list...
Similarly to website questionnaires, emails are one of the fastest ways to reach your audience. Assuming you’re using emails provided by your audience (which really is the only way to go), you can count on incredibly valuable feedback from users who have expressed interest in your service/product.
Similarly to website questionnaires, you can target specific segments of your email list.
A quick way to reach an engaged audience – assuming emails have been provided to you willingly (for ex. newsletter list), your respondents are likely to be more engaged than those who only sporadically visit your page
Effective for sending targeted customer satisfaction surveys
Response rates can be very low if the call is automated
May come across as obtrusive if calls are initiated too often or at the wrong time
Since the introduction of GDPR, and after the famous Cambridge Analytica scandal, respondents have become reluctant to reveal information on the phone.
These used to perform much better decades back before online surveying methods emerged. Nowadays, face-to-face questionnaires have mostly taken the form of meetings via video conferencing software.
As they’re quite time-consuming, today the approach is mostly carried out among random respondents (i.e. on the street), at an incentive to the respondent, or among very specific, carefully selected members of the audience (which, in its own right, might require previous pre-qualifying questionnaires).
Good for modestly-sized feedback collection goals
If carried out on the street, it provides a good overview among random respondents
People tend to be more invested in questionnaires in person, and might be more prone to respond to open-ended questions.
Time-consuming, both for the respondent and researcher
In-person questionnaires with carefully targeted groups often require running pre-qualification surveys weeks ahead.
Structured (question order is predetermined, researchers only go off script to clarify the question or ask for more details)
Semi-structured (questions are predetermined, but the researcher is at liberty to use informal language and tweak wording)
In-depth (questions aren’t predetermined, the conversation relies mostly on one’s subjective point of view/opinion).
Number of participants
In-person interviews: these will be invaluable if you’re looking to read between the lines – especially if you adopt an in-depth or semi-structured approach. In the same way as in the case of in-person questionnaires, respondents are more prone to answer more precisely when addressed face-to-face.
Phone interviews: Similarly to phone questionnaires, the response rates are high – even more so, as interviews can’t (at least, as of yet) be effectively carried out without a fellow human researcher on board. There is a significant difference though – while phone questionnaires remain on script, interviews, with their more open nature, encourage sharing opinions and, as an effect, often collect insights of highly subjective, emotional nature.
While these aren’t necessarily interchangeable with other surveys that made the list, it’s worth noting nonetheless. Focus groups are a way to trigger discussion on a given subject among several participants. It is an amazing way to uncover your audience’s attitudes, as it not only promotes openness but also shows how diversified opinions evolve throughout a live discussion.Now that you know all the survey instrumentation options, the question is:
What are some telltale signs you’d benefit from a mixed approach?
Here are a couple of examples:
You’ve been uncovering recurring behavior within a group, that can’t be explained through correlating other behavioral or attitudinal data.
Your respondents are less prone to answering open-ended questions than they have before and leave more and more empty fields.
Your respondents answer open-ended questions, but the responses are less insightful than you’ve imagined. Also, you don’t know how to formulate a questionnaire that triggers a detailed response anymore.
There’s a very specific user group you’re increasingly interested in hearing more from – ideally, in a less structured, individual way.
You’ve been measuring specific customer metrics via surveys (for example, NPS® survey or CSAT survey, as explained below) but the results have been introducing confusion. You don’t really know what your respondents think anymore – or, more importantly, what they’re actually evaluating.
If you're looking to start surveying your user base, it's best to try digital methods - they will give you maximum results with minimum effort. Whatever channel you want to go for, Survicate has you covered. Your first 100 responses are on us.
NET PROMOTER, NPS, AND THE NPS-RELATED EMOTICONS ARE REGISTERED U.S. TRADEMARKS, AND NET PROMOTER SCORE AND NET PROMOTER SYSTEM ARE SERVICE MARKS, OF BAIN & COMPANY, INC., SATMETRIX SYSTEMS, INC. AND FRED REICHHELD.