1, 2, 3, 4, 5, Everybody in the car so come on let’s write a survey!
Have you ever thought about what goes into writing a great survey? Well here are some tips for you!
This month I am plugging Ranken Jordan Pediatric Hospital. To donate, please visit https://rankenjordanfoundation.org/donate/
The Marketing Gateway is a weekly podcast hosted by Sean in St. Louis (Sean J. Jordan, President of https://www.researchplan.com/) and featuring guests from the St. Louis area and beyond.
Every week, Sean shares insights about the world of marketing and speaks to people who are working in various marketing roles – creative agencies, brand managers, MarCom professionals, PR pros, business owners, academics, entrepreneurs, researchers and more!
The goal of The Marketing Gateway is simple – we want to build a connection between all of our marketing mentors in the Midwest and learn from one another! And the best way to learn is to listen.
And the next best way is to share!
For more episodes:
https://www.youtube.com/@TheMarketingGateway
Copyright 2025, The Research & Planning Group, Inc.
TRANSCRIPT:
If you didn’t know, my day job is marketing research, and I run a full-service marketing research agency called The Research & Planning Group.
But I wasn’t always a researcher.
When I was a marketing student back in the late aughties – by which I mean the first decade of the 2000s – I had to take a class in business statistics, and I was failing it. That wasn’t unusual – our professor warned us that it was a weed-out class that a third of the students would fail and some would have to take multiple times – but it was frustrating because I simply could not understand what it was I didn’t understand about statistics. On the surface, everything seemed to make sense, but I was the sort of person who liked things to be arranged in neat, geometric patterns with hard number answers, and statistics had this whole fuzzy idea of probability that really didn’t make sense to me.
And then, one day, and I really can’t explain this, I was sitting at my desk in class and it all just started to make sense. It was like my brain fundamentally changed in that moment, and I suddenly not just understood statistics, but I had this new ability to see the world probabilistically. Normal distributions suddenly made sense to me. I could grasp the concept of a standard deviation or a p-value with ease. I no longer needed to second-guess myself on whether I should use a mean, median or mode to report data.
It was like I’d had some sort of religious epiphany, except it was in a math class.
I went on to get an A in the class and to realize that maybe I ought to consider marketing research as my specialty. And of course, that’s exactly what I wound up doing. Here I am almost 20 years later running a marketing research agency, teaching marketing research courses and, of course, doing lots and lots of statistical analysis.
But I’ve never taken for granted how hard statistics are for people to understand. One of my favorite books is Darrell Huff’s “How to Lie with Statistics,” a 1954 tongue-in-cheek guide to help people learn how to utilize stats correctly and to see how policy makers, pollsters and journalists often distort them to fit a narrative. Some of those errors are intentional, and some are unintentional, but you know what? I’ve seen plenty of both in the corporate world, and when data aren’t used properly, some really bad decisions can be made as a result.
If you’re the person interpreting the data, you know to avoid those errors. But if you’re the person who’s hiring someone to produce that data and analysis for you? You might unintentionally make some mistakes that could cost you big.
So I’m going to do you a favor. I’ve been a marketing research professional for a long time now and I’ve picked up many tricks I can share with you on how to ensure you get great data and insights from survey research as often as possible without having to become a statistician yourself.
I’m Sean in St. Louis, and this is the Marketing Gateway.
So let me level with you before we get started – The Marketing Gateway is not a college classroom, and I’m not going to use the rest of this episode to try to teach you everything you need to know about surveys and marketing research, because we’d be here for days, maybe weeks.
What I’m going to do is give you five tips to make sure that you can use survey research to its fullest potential. And if you want to know more, reach out to me and I can link you up with some additional resources.
<CAPTION: SURVEY TIP #1: Define a clear purpose for the study>
So here goes. My first tip is to never, ever commission a study without first being able to explain, “what the heck are we trying to learn?”
The more precise this question is, the better your survey will be. And in marketing research we often spend a lot of time trying to figure that question out, because often, what prompts a study is not a true problem, but the symptom of a problem.
So let me give an example. As you can probably tell, I have a cold right now.
It’s making my voice sound a lot deeper than usual.
But how do I know I have a cold and not, say, the flu or RSV or even COVID? I don’t, actually, but the symptoms can clue me in. Do I have a cough? Check. Am I congested? Check. Am I run-down and tired? Definitely. But do I have a fever, or chills, or pain in my chest, or trouble taking deep breaths, or anything else like that? Well… no! And so unless my symptoms start to get a lot worse, I can judge that I have a cold, hunker down and drink some fluids and take some cold medicine to make those symptoms more bearable and know that since the cold’s a virus, there isn’t much else I can do but let my body fight it off.
But let’s say I was having severe pain in my abdomen and a fever and I wasn’t able to keep food down and I couldn’t into a comfortable position without feeling the need to move around because it hurt so much to stay in one position for too long. Oh, and if you poke me in the right side, the pain gets 10 times worse. Sharp internal pain is usually a sign you ought to be heading for the ER, and indeed I should be, because those are the symptoms of appendicitis.
In business, symptoms are often things that aren’t fun for us to deal with. Sales are down is a common one, but so is complaints are up or people aren’t returning our calls or we’re seeing less foot traffic in our stores or website visitors have dropped off. Those are all things that seem like a clear problem that can be addressed by sending out a survey and asking, “why?”
But to a marketing researcher, those aren’t a problem at all. They’re the symptom of a deeper problem that usually requires some thinking to get to. Let’s take sales are down for a moment. Maybe sales are down for our company, sure, but what if they’re down for every company because we’re entering into an economic recession? Then the problem we really want to understand is how we’re doing relative to our marketplace competitors.
What if the reason sales are down is because there’s been a change in the marketplace, like people are less interested in our product or service than they were a few years ago? This happens all the time when product categories mature or begin to hit a decline stage, and asking people why they’re no longer buying isn’t going to yield much useful. But understanding why we’re losing relevance and how we can position our product or service to meet the needs of today will yield a lot.
Defining your problem is so important that I wouldn’t leave it up to one person and I also wouldn’t try to use an AI platform to help you think through it. I’d spend some time talking to anyone you can and forming a deeper understanding of the real issue, and then I’d concentrate on figuring out what you really need to know to solve that problem before you ever begin writing a survey.
Which leads us to thinking about who that survey should even be for!
<CAPTION: SURVEY TIP #2: Define your population clearly>
If you want a survey to be effective, you need to think very carefully about who is going to be taking it. If you define your survey as “general consumers Age 18-65 from the United States,” you’d better prepare yourself for some uninteresting findings, because general consumers are so varied and unlike one another.
Just on my street where I live, where everyone’s in roughly the same socio-economic cohort, we’re got a pastor, a retired couple, a mortgage closer, a UPS driver, a respiratory therapist, an academic adviser, a used car salesman and a government contractor who used to be in the military. And those are just the neighbors whose profession I know! Some of us have kids. Some of us don’t. Some of us are married. Some of us are single. One neighbor, I know for certain, is a widow. We’re all really different despite sharing the same street in our address.
You know those jokes that start off, “A priest a rabbi and a podcaster all walk into a bar?” Be prepared to add a dentist and a sanitation worker and the head of IT and a college student and a beautician and a fry cook and a bus driver and a stay at home dad and a professional yo-yo champion and a guy who’s couch-surfing at his friend’s place, as well as a whole bunch more people, in a sample that’s going to have a lot of opinions, sure, but from a throng of people who are unlikely to be anything alike.
So the key is to spend some time defining your population, not just demographically but also categorically, behaviorally and psychographically as well. If you’re conducting a survey about a product for moms of young children and you aren’t defining your population as “females who are 25-50 with at least one child aged 18 or below living at home,” you’re going to get a lot of folks who don’t fit the group of people you’re looking for. And I would further suggest screening for things that matter to your campaign, like shopping at the places your product will be available or paying attention the media you’ll be using to advertise or having an interest in your category to start with.
This will not only help you talk to the right people, but also ensure that the sample plan you create to reach those people will be more effective… though, I will warn you, it also may mean it’s more expensive and requires more expertise to get to.
<CAPTION: SURVEY TIP #3: Design questions that tell you only what you need to know>
So once we get that purpose and population in place, my third tip is to not overcomplicate things with a design that looks like something you’d take in a psych class – one of those surveys with 100 questions you have to rate on a scale that put all these strange scenarios in front of you.
Most surveys only need very simple lines of questioning. If your survey is quantitative, your question formats are generally going to be multiple choice – that’s what we statisticians call nominal data – lightly ordered – what we call ordinal data – or set on a scale with equally spaced points – what we statisticians call either interval or ratio data depending upon the type of scale.
Of the three, nominal questions are the easiest to understand, but also the least useful for more complicated forms of analysis. Here’s a good example of a nominal question: “Which of the following best describes your gender?” There’s no better or worse in that question – just choices.
Ordinal questions, where you put things in an order like most to least or less interested to more interested, can be used for a little bit more, but they’re still not as useful as numerical questions where you have something like a 5-point, 7-point or 10-point scale where things are being rated.
An ordinal question might be something like, “How interested are you in this product?” with the choices being, “Not interested, somewhat interested or extremely interested” where the order implies a progression without really defining how much interest changes between the data points.
A numerically scaled question would instead explain that you want them to assess their interest on a 1-5 scale and explain what the endpoints mean. If the scale is balanced, you’d have a neutral midpoint at 3. And if the scale is not balanced, you’d simply explain that one of the points – let’s say 1 – means “I’m not interested” and a 5 means “I’m very interested” and let the person decide for themselves whether that interest is a 1, 2, 3, 4 or 5.
You can do a lot with a numerical scale in analysis because it lets you perform a greater number of statistical processes on the data, like calculating a mean or collapsing the scale down into categories or running a multivariate predictive analysis. It’s cool stuff!
But even if these number scales are preferable for analysis, they’re also harder for survey respondents to understand because you’re asking them to translate attitudes to numbers.
So, the best practice is to ask easy multiple-choice style nominal or ordinal questions when you want to ensure your participants understand what they’re answering and to ask numerical questions when you can tolerate a little bit of uncertainty. And even then, it’s a good idea to ask a pro for help before you get big ideas about analysis.
<CAPTION: SURVEY TIP #4: Keep question wording simple and focused>
I love words, and I love to write, so you’d better believe I’m great at writing long-winded pieces of prose that are so beautifully constructed that you’ll completely forget what you were thinking about when you started to read my writing, and maybe instead fall into a comfortable reflection on the journey along the way.
That’s a great skill if your goal is to write a novel, but terrible if you’re trying to write a survey. A good survey needs to use as few words as possible while still providing clear instructions for what it is you want people to respond to.
One tendency a lot of people have when they’re writing surveys is to want to ask questions that fit a certain style or marketing philosophy. Don’t do this. It can make participants feel like you’re trying to sell them something and can even result in their feeling led to give you more positive answers because we human beings tend to have a bias towards telling others what we think they want to hear.
Another thing many people tend to do is construct questions that are double-barreled, by which I mean they ask about two things instead of one. “Do you prefer a beer that tastes great and is less filling?” is a double-barreled question, because the meaning is unclear. Is the respondent reacting to the “tastes great” more than the “less filling?” Or vice versa? Do they care about one thing or both things? Or do they only care about both things if they’re together? We don’t know. The rule of thumb is that if you question has the word “and” in it, check to make sure you’re not actually asking about two things. If you are, split it into two questions!
One more thing to watch out for is jargon or academic language or terms that need to have a trademark symbol after them, because they’re off-putting to survey respondents. If you need to use this kind of language, use it sparingly. Acronyms should only be used if they’re defined and if they make sense to the reader because they’re in common use. For registered trademarks, I recommend putting one note in the beginning or end of the survey that your trademark exists and dropping the little TM or the R in a circle from other mentions of your brand. It makes things less confusing.
The last thing we want to do in a survey is confuse people when we’re asking them to give us information. And we also want to avoid annoying them if possible.
<CAPTION: SURVEY TIP #5: Use yes/no questions as a tool, not a crutch>
This brings me to my final tip: try to use yes and no questions sparingly. For one thing, they’re very limited in what you can do with them. But for another, there are very few situations in life where people can confidently answer yes or no without feeling to a need to explain why they gave that answer.
“Are you going to the movies this weekend?” is relatively easy for people to say yes or no to because most people don’t go to the movies that often anymore.
But “Would you consider going to the movies this weekend?” is tougher, because they might consider it even if they’re not planning to go.
Likewise, a question like, “have you ever been to Chicago?” is easy because it’s a fact-based question. But “Do you believe you’ll go to Chicago sometime in the next five years?” might be more difficult, especially because people travel through Chicago all the time on airlines and they might be thinking that’s good enough for a “yes.”
Where yes/no questions are best-employed are as what we call filters that are designed to shorten the survey for people for whom questions aren’t relevant. So if we asked people, “Have you ever been to Chicago?” because we have a lot of Chicago-based questions for them and we only want to ask them if they’ve maybe been down the Magnificent Mile and strolled along the Navy Pier and visited Wrigley Field and had a Maxwell Street Polish dog and indulged in some deep dish pizza at a place off Wacker Drive, then a “yes/no” question can save them some time and aggravation, because we won’t ask those questions if they say no. It’ll also make our data analysis easier, which is always a plus.
I hope these tips have all been helpful for you! And remember, if you need to call in the pros, a seasoned marketing researcher like myself can help you think through all these things and more to get the information you need to move forward.
I’m Sean in St. Louis, and this has been The Marketing Gateway. See ya next time!
PLUG
This month’s plug is for Ranken Jordan Pediatric Bridge Hospital – and no, I don’t have any connection to them even though we both have Jordan in our name. It was actually founded in 1941 by Mary Ranken Jordan, an Irish-American philanthropist whose uncle, David Ranken Jr., founded Ranken Technical College.
Mary was very active in the St. Louis area along with her husband, an accomplished musician and cutlery industry manufacturer named Clay Jordan.
Again, no relation. But I’m happy to share a surname with these fine folks!
Ranken Jordan operates a 60-bed health care facility in Maryland Heights designed to help infants, children, teens, and young adults aged 0–21 with specialty care pediatric needs or who require services such as rehabilitation or interdisciplinary care. They operate under a bridge model to allow pediatric patients to receive care both inside and outside the hospital under close medical supervision. They don’t turn children away, even if they can’t pay for services, and they emphasize finding ways to encourage and empower children with positivity and finding ways to say YES when a child wants to do something.
Oh, and they’re just one of a few pediatric bridge hospitals in the United States, and the only one in Missouri!
You can learn more at https://rankenjordan.org/, and they accept donations through the Ranken Jordan Pediatric Hospital Foundation at rankenjordanfoundation.org/donate/
Leave a Reply