Is your Advertising Campaign Effective?

Checking the Pulse –

I wasted almost an hour of my time the other day with a grumpy guy who works in the yellow fats market (butter/margarine). He talked about the various ‘segments’ of his market. He talked about what the ads we showed him meant. To him. He debated the ‘premise’. The ‘brand values’. The ‘take-out’ of the ads. He didn’t get the ads or the strategy at all. Funnily enough, he was not the target market.

This is why people often commission market research. To get around an individual who’s influencing their organisation, who’s ego is flying in the face of logic or ethics. Who may have a hidden agenda, or is just plain dogmatically driving down the wrong road fast. You have a professional obligation to do the right thing by your company and it’s brands. You have a right, no a responsibility, to undertake research when you think it’s appropriate. And it’s your responsibility to make sure the board acts on the results.

I have spent about 20 years of my life asking people why they do one thing or another. After a while a pattern develops. You can slot them into boxes and draw conclusions. If you have brains, you can use those conclusions to make (dare I say it) recommendations. That’s market research.  It ain’t complicated. Don’t let people con you that it is.

The theory behind market research is that if you know how your customers feel about your widgets, or even who they are, you’re somewhat better off than if you sail blindly into the sunset making the same mistakes over and over again. I’m not so sure even this basic premise is achieved very often.

The problem with a lot of market research is that while it is probably the most vital thing you can do, it is also inherently inaccurate and over-done. It’s usually practiced by people with little knowledge of customer behavior or the workings of the commercial world, (anyone with an English accent and a Masters in Psychology has to be looked at with suspicion) and most of the time it’s a snap shot, so it’s not as relevant a few months later and the results are almost never repeatable, like real ‘science’ is supposed to be.

Keep this in mind. A board can only act on a total of about five recommendations from any market research exercise. It’s only the indisputable, obvious stuff they will do. This is not to say an extra bit of information isn’t useful sometimes, it’s just that most of the time it’s a red herring, and inexperienced researchers or marketing managers get held up on these herrings and miss out on the main issues. The big fish.

It’s the interpretation of the findings relating to the big issues that you should pay for. It’s the researcher who will say to you, look, the results say X, Y and Z, but I think you should read them with a touch of salt and do A, B and C instead, who’s really worth their fees.

Like any commercial exercise, you don’t want to deal with someone who comes back with a 90% fit from the brief. It may be a good pass at school, but it’s a bad failure in life. You want to get a much better return on your investment. You want someone who adds an extra 50% on top. You want insight, and art, not a photograph. Any dickhead can take a ‘photograph’ of a market. What you’re really looking for is the interesting, useful fact that you can actually act on, that will translate into much higher sales or a better customer retention rate etc.

How to do effective market research

Most people commission research because they don’t trust the information they are getting from their field teams or they’re unsure about what their ad agency is trying to push them into. Below I’ve assumed that you want to do it cost effectively and that you may consider doing some of it internally. I’m all for companies who pick and choose what they out source and what they’ll get their own people to do.

Quantitative vs. Qualitative

Quant research, relates to large numbers of interviews or Quantity. Qual research relates to usually small numbers of respondents from whom you get real depth of information or Quality. Use either or both according to what you need to know.

Qual Methods

Focus groups

Focus groups are the quick, dirty method favored by lots of marketers for simple answers to issues. They are a mainstay of ad-land for testing creative, attitudes etc. as it’s pretty easy to argue with the results if you need to, and you can do a group and still squeeze in an hour or so with the bit on the side before you have to get home to the spouse.

They are usually 8-10 people in a room discussing a subject for about 70-100 minutes. For most problems, you’d do one or two groups per market segment. Older women, younger men, Sydney versus Melbourne or whatever. So most serious studies are of say 6-10 groups. To test an ad, you might only do 2-4. Focus Groups will set you back about $4,000 bucks a piece if done by someone credible.

Focus Groups are great for medium-depth understanding and can be run to cover a wide variety of issues relating to a subject. They must be run by someone who can control the participants (you must not let one leader type force the others to change their own opinion) and work them to get the answers you need. As a client, you usually sit behind a big two-way mirror and watch the exercise. A good ‘moderator’ will come into your room a couple of times during the session and ask you what additional issues/aspects you want to explore.

It’s felt that you get more depth about a subject in a focus group as versus an individual interview, because people debate/argue a point. This is true. If you want really effective focus groups, do them, if at all possible, where the purchase decision is made. Say a supermarket. It’s very hard on the researcher, but the results are dramatically more accurate.

As they are almost always video taped, it’s a nice trick to get the good bits on the tapes edited together into a 10 minute presentation. It’s pretty hard for people to argue with strangers saying something on the box, regardless of how statistically accurate those findings may be.

Affinity Groups

Small focus groups, usually with people from the same family or work team. At almost the same money, definitely not cheap. (They take the same amount of researcher’s time.) But good for occasions where you have complicated purchases or attitudinal circumstances etc. Like say those in the company who get involved in buying a new phone system…

One on One

The original and most flexible of all research methods. At the going rate of at least $300 per interview, expensive per respondent. Used mainly for industrial research where you’d have only a few (20-200?) to interview, or for complicated purchase decisions where you need to get to the bottom of an issue. I’m a big believer in using one-on-ones in small numbers just to hone directions before venturing into another methodology, especially when you’re polishing a questionnaire for a big survey.

Quant Methods

Except for some small studies in industrial markets, this is usually not an area to do yourself. Get some serious help from a quant company, at least for designing the questionnaire and the cross analysis (How did the males 20-25, with kids, who did not have credit cards, but had bought a car in the last 12 months, respond to the issue of …) or you’ll find yourself going nuts trying to analyse the data.

When you get the telephone book sized report back, have your own people to wade through the data. You’d be amazed at how much extra information you can get.




  • are short/user friendly
  • set up the respondent for the issues in a positive, friendly way
  • don’t manipulate the results
  • cover the critical basics
  • kill off uncertainty – by cross checking questions
  • give you detail
  • are easy to analyse
  • allow you to  assess results by useful respondent categories


  • are short, accurate, definite
  • can be used by your organisation to make a financial change to your group
  • have results that should have been picked up by your other systems
  • add something no-one guessed
  • make sense to the reader


Door to Door

Some pimply kid with a I.D. on their shirt knocks on the front door to ask the resident a couple of questions. Most of the time they have the door slammed in their face. Not an easy market research area. Hardly anyone does it anymore. Shame. It’s quite cost effective and a pretty accurate way of gathering quantitative data. If you’re doing it yourself (and most market research can be done by your own people if they are well managed) you’ll need to make sure the team have convincing looking I.D. and lots of gumption.

Shopping Centres/Strips

One of my favorites. You can interview hundreds of people a day in the right location, such as Southland (who do tend to sting for ‘short term leases’ something chronic.) No, you can’t get a lot of depth out of a person if they don’t have the time, but you can show ads, get feed-back on new product designs or what it is you need, in a better manner than many of the more traditional research methods. You usually get a little market research booth, power and the right to roam freely around grabbing people with an offer like “If you just answer these questions, I’ll give you a chocolate (or a $20.00 voucher for Sanity Records)” etc.


You’re half way through the carbonara and the phone rings. “It’ll only take 2 minutes”. 20 minutes later, your dinner’s cold, you’re severely pissed off and you’ve spent half of the phone call making up numbers or quotes just to throw the researcher. So does everyone else. Phone research may be the basis of much of the political and other quantitative research, but I’ve got to tell you, unless it’s well worded, quick and intelligent, you are almost guaranteeing much of the information is completely inaccurate because the method has put off the participants. How to get around that? Don’t force the telestaff to use a standard opening line. Tell the truth when they ask how long. well, And if it’s too, long, shorten it. Better to run three short studies that get decent results, than one long survey where you suspect most of the latter answers are bullshit.

Snail Mail

Occasionally done by government departments, the census is an example, but have tended to be over-run by more modern forms, such as on-line research. Big juicy surveys can give you lots of data if managed well, but they are very expensive to run as someone has to key in the data and they are deadly slow for obvious reasons. But worth it if you have a handy data-base and can give people a real incentive to fill out the questions, like say a hefty fine, in the case of the census.

Paperwork – Application Forms/Warranty Cards

The best time of all to get data on people and their attitudes is when they have a logical reason to give you the info, such as when filling out a warranty form or their details to make a claim etc. Mad not to use the opportunity when it presents itself. Can even be very useful to stick questions on products and ask them to send in comments without any other incentive. You’d be surprised how many people don’t mind helping out a company if you’re just plain honest about why you need it.


Fantastic. Fast. I mean you’ll know the answer in minutes or hours. Can touch on all sorts of subjects. With PDF’s of brands or movie player versions of ads, it can be wildly effective ad testing. Often can be a bit weak on detail. Sometimes can’t get at the real reasons behind an attitude, and hard to get a truly representative survey, given there’s only people with computers writing back, but at almost zilch cost, who cares if the information isn’t absolutely perfect?


Online is the Eldorado (Mountain of Gold) of market research. It’s quick and easy to change questions. You can show different images, different deals, to different people. Can be cut into little surveys and run almost constantly. The only things to watch for are decent numbers of respondents to keep the survey reasonably statistically accurate and to check where the answers come from. There’s little point in looking at survey results for a women’s product when half the answers have come from men who felt like responding so they could win the telly.

Choosing a method (s)

As you can see from the above, there’s plenty of options to gather data. Some are more accurate , some are cheaper. Most people selling research will be comfortable with only one or two methods. These are the ones they’ll suggest. Knowing this doesn’t make the decision any easier, but it does get you past the issue of whether you can trust their advice. You can’t. Just like anyone else in marketing.

Cheaper Options

Many of the above methods can be bought in an ‘omnibus’ form, where your questions are put in with other companies in a big survey, or ‘syndicated’ where most of the switched-on players in a market pool the costs and results of a study.

How to do accurate

Unfortunately accuracy depends on one basic issue. Numbers. And usually numbers mean money. If you get enough people to answer a question and make sure you have a few cross-checking  questions slipped in to weed out the liars, all you need to do is make sure you’ve hit say 700 people in the right demographic to give you iron-clad results.

Get the basics right

Almost nothing in marketing is as vital as knowing who you’re talking to, what to say and what they think of you. Virtually none of the market research in this country addresses these fundamentals. Why? Because often as not, everyone assumes companies already know it. Including the company themselves. If you get the 20 or 30 simple things that all businesses need to know right, you’ll be doing better than the 90% of marketing managers who can’t answer the basic questions.

Commissioning Researchers

Decide what it is you need to know, how accurate you need the answer to be, how best to find it out, brief two or three research firms and sit down when they give you an estimate of cost. Choose the firm who sends you the shortest, easiest to understand quote and the most likeable executives. I personally like people if they suggest useful ideas.

Please note a PHD (Dr. in front of their name) does not mean a thing to you. For a start, they got their PHD for a very close analysis of a usually pretty obscure subject. If their area of study is not highly relevant, you may as well be talking to a three year old for all they know about your business. Fine to get them to do the maths. Mad to listen to their recommendations.

Ask them about relevant experience. What they found out and how was it used. If they flop around, pick up the Yellow Pages and get another mob in.

Move the goal posts

Market research is expensive. When it becomes bloody obvious you have found out what you need, it’s quite smart to cut the number of groups, or to change the focus of the research.  Say you’re doing phone research, you’ve already done 200 interviews and  80% of the respondents think X. There’s little point doing another 500 interviews to confirm X. If you’re committed to the budget, find out Why X, How X and what else you can sell with X etc.

Send the Report Back

If they send you a bunch of numbers strewn across the page you can’t understand, send the report back and ask them to produce something you can use. And if they send you a phone book sized report with no information you can use in it, tell them you won’t pay the bill until they’ve done it in a way you can get useful data from. Don ‘t take that bullshit about ‘that’s how the industry does things’ or whatever other patronising crap they come out with. Researcher’s have the same obligation to be user-friendly as every other profession in the country.

Give them less, not more

Let’s assume you’ve got a report you can understand, you’ve had a good think about it and worked out what the findings really mean to the company. When you’re presenting your market research findings to the board, cut them down to only about 10 key findings, with only about 5 simple recommendations.  And make them absolutely inarguable.

That way, you’ll get the respect of the board for your ability to treat their time as valuable. Then you just have to live in vain hope their old, male, soggy, accountant brains can process the information  . . . and you may even get what you want.

Market research is often plain wrong

There’s the old story of the micro-wave study where the respondents loved the idea of blue, green, pink etc. microwaves. When they ended the interviews and gave them a choice of what color to take home, they all chose white. Happens all the time.

How would the bugger ad have researched? You can see it, can’t you? “What’s your understanding of the ad?”  “Oh, if you buy a Toyota, you’re always saying bugger. And it’s the car that rips cows heads off…” That’s why I say use people with brains to interpret your research. There’s a time when you listen to the public, there’s a time when you hear what they say, but your experience tells you what they’ll really do. A reseacher who has the balls (sorry, ovaries?) to argue with own findings is worth paying serious money for.

Leave a Comment

Your email address will not be published.

Scroll to Top