Quality in Market Research – history? Or hope for a paradigm shift in 2018?


What will 2018 bring in the world of market research? Many of the world’s largest economies are growing, stockmarkets are at historic highs, so potentially fertile conditions for investment and MR growth?

If you read the MR predictions for 2018 on the Greenbook blog, say, or Bob Lederer’s RFL feed, leading MR industry voices are often quite cautious, talking of winners and losers. Many hope that 2018 will be better than 2017, which was a challenging year for many in research, with large clientside companies tightening their approach to budgeting, expecting and getting more research done faster and cheaper.

Is this a sustainable trajectory? Many of the responses to agility will be from the world of tech. – automation, AI and more.

But what about quality – of participant experience, analytical rigour, storytelling abilities, with or without video? Tech can take us so far, but we’re in charge of design, inputs and outputs.

More critically: can the thinking required to make insights inspiring and challenging keep pace?

Maybe we need to change the way we look at things. I’d love to make 2018 as the Year of MR Quality. Three reasons, here are the first two:

Firstly, there is plenty of evidence to suggest that the automation and replication of good MR practice via DIY software isn’t happening on a broad enough scale. More research is resulting in huge quality variance.

Second: we have a duty to protect and nurture the MR space from tech-savvy innovators, often with minimal if any social science background or expertise.

Before I list the third reason, maybe worth asking: is quality a lost MR cause? A crusade resembling a Don Quixote-style undertaking?

Jeffrey Henning seems to adopt a fatalistic attitude towards “quality” in his Greenbook blog prediction for 2018 ( Poynter’s earlier pithy 2017 prediction on the MRS UK portal was in a similar vein:

I’d say quality is a cause worth fighting for – shifting to my my third reason for a quality push: quality perceptions are linked to value, budgets, pricing and margins, so are central to our future well-being.

We all need to raise our voices – #qualitymetoo style.

Here are my thoughts on what can be done in 2018, clientside or supplier.

1. Challenge and call out bad research, whenever it breaks.

We have all sat through flimsy papers at conferences that were over-generalising, making broad-ranging claims supported by a thin body of evidence. Suffer Dunning-Kruger-style stuff in silence? C’mon guys.

The onus is on all of us to all call out bad research and associated concepts – the phrase “instant insights” is one of my bugbears.

Social Media is a great sharing platform – Linked In for example. There’s no need to name and shame, it’s easy to sanitise, avoiding being too personal or confrontational. Ray Poynter is someone who does this well.

2. Be a Quality Stickler – adopt a Zero Tolerance Approach.

Bad quality in MR can be very simple – ridiculously small sample sizes, leading questions, inappropriate use of the question “why” in diagnostics, poor rating scales, unmanageable grids….a few words to the better can help.

Reminding folk of more sophisticated stuff, such as the dangers of confusing correlation with causality, referring to spurious correlations in connection with Big Data analytics is fine – but sometimes simpler stuff can be more effective as per the examples above.

However we do it, employing a zero tolerance level to bad practice in all shapes and sizes can be effective.

So try being a quality zealot – in a charming way of course ;)

3. Highlight the risks associated with poor quality Research.

DIY MR software is a great enabler – in the right hands. But how many of those using DIY are focussed on the “how”, rather than the “what next” questions? Most business people are focussed on outcomes, speed, cash-flow, financial metrics, churn. Customer centricity is a concept learned at business school, alongside NPV and others – when not properly internalised and utilised, it can have fatal consequences.

An example: 80% of start-ups still fail, and part of that is related to a lack of real consumer need appreciation.

We need to open up a mental MR divide between good and poor, and show the risks, sensitise to the downside, linking cheap, quick and dirty research with less-than-successful business outcomes, repeatedly, insistently, to juniors, seniors….the customer is queen. Period. Save money there and you can downgrade your forecast pronto.

The nitty gritty? This could be respondent fatigue – too many concepts per participant for example, or response quality as a function of interview length, poorly worded questions. Just make the link to the business outcome, don’t sound academic or defensive.

Sanitised negative case studies might be something to consider to show budget owners the risks of scrimping and saving on the MR budget. Esomar – an idea there? IP rights waived, btw ;)

4. Bring your Social Science expertise to bear – but think like a Business Owner.

Many of us have training and experience in sampling, statistics, significance testing, know the value of accessing derived importance, are familiar with a multitude of cognitive biases – we need to highlight and leverage these skills, weaving these quality credentials into our everyday work, fusing the practical with the more intellectual.

It raises our game in the eyes of others – and works against commoditisation.

It’s something to practice: sounding knowledgeable in a grounded manner. If our start point perspective is through a business lens, then supporting MR technical knowledge will likely be appreciated rather than seen as dry, academic.

5. Constantly talk Quality.

Esomar member? Making use of that in your collateral? ISO certified? In-house your recruiting? Special quality criteria that set you apart?

Quality is a huge concept – which means there’s lots of angles, lots to talk about to make it come alive.

Above and beyond your brand, there are multiple small quality helpers that if communicated consistently can help your perceptual standing, signify “expertise”, and open up a gap to all sorts of business folk that have little or zero social science or MR training.

Behavioural economics continues to be a huge filip for MR rigour – Mr. Kahneman has done his bit, the same can be said for Mr. Thaler in 2017. Time for us lesser mortals to pick up the slack on BE, and anticipate what the next big thing might be. Evolutionary psychology maybe? Certainly makes you want to google it fast.


In summary: I’d love for 2018 to be a year where we unhook a bit from tech, calm down and refocus, re-invest in the thinking substance of MR, not just trying to figure ways to automate, cut costs.

It can be exciting – whatever spin you choose to put on quality, if it’s stimulating and tangible, it becomes a value booster, we all stand to benefit.

I’ll raise my glass to 2018 as a year of quality. Cheers.

Curious, as ever, as to others’ views.

Happy Thinking People GmbH
Blumenstr. 28, D-80331 München
Tel: +49 89 290040 4000
Fax: +49 89 290040 4027

Happy Thinking People GmbH
Rosenstr. 17, D-10178 Berlin
Tel: +49 30 327986 8600
Fax: +49 30 327986 8620

Happy Thinking People AG
Stauffacherstr. 101, CH-8004 Zürich
Tel: +41 44 204 16 26
Fax: +41 44 204 16 27

Happy Thinking People Pvt. Ltd., 201, Sai Nara Building, 21 North Avenue,
Linking Road (Above Kotak Mahindra Bank), Santacruz (West), Mumbai, 400 054
Tel: +91 226 6905400
Fax: +91 226 6905454

Happy Thinking People France SAS
20, rue des Capucines, F-75002 Paris
Tel: +33 1 53 66 60 95
Fax: +33 1 46 27 05 35