15 Nov BETTER DATA FOR BETTER RESEARCH RESULTS

At the end of October, BAM organized another edition of its “Meaningful Marketing Talks” at the Kruitfabriek in Vilvoorde, this time on privacy and ethics. One of the noted speakers there was Géraldine Proust of ESOMAR. She talked about the importance of using unbiased and representative data in market research.

ESOMAR is a global business community of insights and analytics founded in 1947. The organization has more than 8,000 members including 650 companies from 130 countries. ESOMAR is a strong advocate for following the statistical and analytical techniques of the applied social, behavioral and data sciences when doing market research. And that is precisely what Géraldine Proust came to reiterate.

“Unbiased and representative market research doesn’t just happen, it’s a continuous industry effort,” she says. “At ESOMAR we set guidelines that ensure that data is processed in an ethical and legal way that protects the rights of research participants and data subjects. Now AI is increasingly used in our field, using correct data is becoming increasingly important.”

Secondary data

This especially plays a role when market researchers use so-called “secondary data”, Proust argues. “Secondary data are data that you didn’t collect yourself and thus come from an external source, say IoT-systems or social media. This data should not be deployed without asking fundamental questions about its provenance: was there a thorough assessment for the first collection of data? Was the data collected with the knowledge of the participants? The role of the researcher is increasingly evolving from data collector to data curator: he or she must focus much more on organizing and integrating the data.”

For this reason, ESOMAR and GRBN (Global Research Business Network) developed a guideline on processing secondary data. This guideline highlights the need for transparency about data sources and includes a set of requirements. These enable clients and other data users to make informed judgements about the validity of the data and whether it is representative. The requirements also cover methods of statistical analysis and, where relevant, methods to clean the data.

In addition, ESOMAR has created a “Data Protection Checklist” to help researchers ensure they have covered the fundamental requirements when collecting and using data. For instance, there must always be a legal basis for processing personal data (through consent or a contract, for example) and personal data must always be as up to date and accurate as possible. Proust: “Data must of course also always be kept secure and there must be clear internal rules around how the data is used. If, for example, cloud services are used to store the data, the risks of this must be clearly identified and kept under control.”

The guidelines can be found at www.esomar.org/codes-and-guidelines

Questions to help buyers purchase online samples

ESOMAR has also drafted a set of questions that buyers should ask when evaluating the offerings of different online sample providers (samples are a selection of participants to take part in a research project). The original set of questions was made in 2006. It was updated in 2012 and the most recent version was released in March 2021, it compiles 37 questions. “What information about a project is given to potential participants before they choose whether to take the survey or not? How often can the same individual participate in a survey? How does this vary across your sample sources? What recruitment channels are you using for each of the sources?”, gives Proust as examples. “But also: what form of validation do you use to ensure each participant is real, unique, are who they say they are? How can participants provide, manage and revise consent for the processing of their personal data? What support channels do you provide for participants? This guide provides the sample buyer with a framework to evaluate the services of the sample provider, offers questions that should be asked and indicates the issues buyers should expect to be covered in an answer.”

More than a checklist, the questions should be used as the basis of a conversation between the buyer and the seller, says Proust, depending on what it is important for the specific buyer. “There are generally no right or wrong answers to these questions, the answer depends on the context and the kind of research. This is all about the quality of exchange and how it is working and helping people to become more aware of what their options are, what is important and what is not.”

Five principles for AI

As Proust pointed out, especially in the AI era, it is necessary to thoroughly define the methodologies of investigations. Accordingly, ESOMAR launched discussions to develop its own ethical framework on how to use artificial intelligence at its most recent conference. “That framework is based on five major principles,” she explains. “The first is that AI should be used for good in market research. For example, to provide an unbiased, representative voice. Or to support that research is conducted ethically. The second is that AI systems should do no harm: we want to avoid discrimination and deep fakes and be transparent to individuals. The third principle is that there should be human oversight of AI systems. It is essential that there be responsible human beings for the organization, use and protocols around AI. The fourth principle is that AI should promote prosperity and avoid unfairness. Training data should be unbiased and representative of target groups. This can be done through correct methods of obtaining the sample. Last but not least, AI systems must be explainable, and developers and users must be held accountable for AI outcomes. AI should be verifiable, so the use of ‘black box’ systems (where no one can really explain how they work) should be avoided.”

Best practices to measure demographics

To finish, the last tool ESOMAR developed, was a series of evidence based best practices to measure demographics in a uniform way. Proust: “ESOMAR experts take into consideration data protection and privacy, while developing these best practices, for example data minimization for age as we do not encourage collecting the date of birth. Our recommendations will improve quality standards and produce results that are more comparable from one project/region to another thus facilitating multi-country research projects. This will help researchers and clients everywhere to improve research standards. ESOMAR consults the broader industry in its guidance and best practices before adopting them, so everyone is welcome to share opinions and suggestions with the professional standards team.”