Artificial intelligence (AI) is revolutionizing the survey and market research sector, introducing advanced tools at every stage of the process. These innovations promise greater efficiency, automation, and accuracy, but they also raise important ethical and methodological questions. In this article, we will examine the current technical and methodological limitations, the main ethical issues, and the existing guidelines and regulations (from the GDPR to the European AI Act, up to the ESOMAR guidelines) for the responsible use of AI in research.
Current technological and methodological limitations
Despite the promises, today’s AI applications in surveys still have significant limitations, both technological and methodological:
Model bias:
AI systems learn from the data they are trained on and inevitably inherit its biases. Generative language models can reproduce cultural biases or stereotypes present in their training texts, risking the perpetuation of these distortions in the questions or analyses they produce. For example, an AI could phrase questions differently (or less appropriately) for certain demographic subgroups, reflecting preconceived notions embedded in historical data. This poses a serious methodological problem: how can we ensure AI is fair and representative? Even with fine-tuning techniques, LLMs may fail to accurately capture the nuances of opinions from underrepresented groups. Researchers therefore need to implement rigorous checks and validations to detect and eliminate bias in AI-generated content. In both academia and industry, approaches are being explored such as using multiple AI models in parallel (including open-source models) to compare results and reduce the risk that the biases of a single algorithm influence conclusions.
Data quality and meaningfulness:
AI cannot work miracles on raw data—the principle “garbage in, garbage out” still applies. If collected data are low-quality or not representative, even the most sophisticated algorithm will produce unreliable results. For example, incomplete or unbalanced training data will limit a model’s predictive ability and may introduce systematic errors. It has also emerged that synthetic responses generated by LLMs (when used to simulate respondents) tend to be too uniform and “clean” compared to real human answers, losing the rich variability that characterizes genuine qualitative data. This means AI can struggle to replicate the complexity and messiness of human behavior, risking an oversimplification of complex phenomena. From a methodological standpoint, having solid and relevant datasets remains essential: guidelines also encourage balancing data minimization (to protect privacy) with the need to use sufficiently large and diverse datasets to train reliable models, ensure fairness, and reduce bias. AI enhances analysis but does not replace the need for high-quality data and robust research designs.
Algorithmic opacity and reproducibility:
Many AI systems, especially those based on deep learning, operate as “black boxes” whose internal functioning is not transparent. This creates opacity issues: researchers cannot always explain why an algorithm reached a certain conclusion or on what basis it segmented respondents. Moreover, the output of an AI model can vary over time: for instance, updating or retraining an LLM may change the generated responses given the same input, making it difficult to replicate a previous result exactly. This conflicts with classic principles of scientific replicability. If an analysis is driven by a model that is unstable or poorly documented, it becomes hard for other researchers to verify or reproduce results over time. Even operationally, opacity complicates compliance and approval: how can you reassure a client about the validity of an insight if you cannot clearly describe the process by which it was obtained? Organizations will need to invest in explainable AI solutions and in documenting algorithmic processes (model versions used, parameters, training data) to improve transparency. In some cases, it may be necessary to avoid overly opaque models where full explainability is required. In any case, it is essential to account for these limitations and use AI cautiously, maintaining rigorous human and methodological oversight.
Ethical issues in the use of AI in surveys
Beyond technical limitations, the widespread use of AI in research raises major ethical issues that professionals must carefully consider:
Protection of personal data:
Surveys often collect personal data (demographics, opinions, behaviors), and AI adds another layer of attention regarding data protection. The key principle is that AI technologies must fully comply with privacy regulations such as the GDPR. This means adopting adequate security measures, minimizing the data processed, and anonymizing or pseudonymizing information where possible. GDPR compliance is not optional; it is necessary to ensure integrity, transparency, and accountability in research practices and to protect data subjects’ rights. Those conducting AI-driven research must ensure respondents’ personal data are not improperly exposed to external or insecure AI systems. For example, if third-party cloud AI services are used to analyze responses, it is crucial to verify where data will reside and how they will be used (preventing data retention or model re-training without consent). A practical recommendation is to avoid entering sensitive details or proprietary information into AI tools whose data usage policies are unclear. The security and confidentiality of participants’ data is an ethical imperative, even before being a legal one.
Informed consent and transparency toward participants:
Ethical AI use requires maximum transparency toward research participants. People must be clearly informed about what data are collected, for what purpose, and (when relevant) whether automated technologies will be used in procedures. The GDPR requires consent to be freely given, specific, informed, and unambiguous for the collection and use of personal data. In survey contexts, the consent form should explain whether responses will also be analyzed using AI algorithms and for what purposes, so that participants are fully aware. In addition, respondents retain the right to withdraw consent and to be “forgotten,” meaning they can request deletion of their data at any time. A specific transparency issue concerns the declared use of AI: it is good practice to communicate (at least in general terms) when parts of the research process are automated. For instance, if a respondent interacts with a chatbot, it should be clear that they are speaking with a virtual assistant and not a human operator. Similarly, reports to clients should indicate if certain analyses (e.g., consumer clustering) were performed using machine learning algorithms. International guidelines recommend always making explicit when and where AI is being used, for example by clearly labeling automatically generated text or content. This transparency builds trust and ensures participants give consent with full awareness.
Accountability and human oversight:
A central ethical dilemma concerns who is responsible for errors or negative consequences resulting from AI use in research. If an analysis algorithm misclassifies respondents or generates misleading recommendations that lead to poor decisions, who is at fault? It is essential to establish that ultimate responsibility always lies with the researchers and organizations using AI—not with the machine itself. This translates into the need for active human oversight: AI must operate under the supervision of professionals who verify outputs and intervene when anomalies occur. Best practices emphasize a “human-in-the-loop” approach: humans should remain involved throughout the AI lifecycle, both during development (validating the model, setting ethical principles) and during operation, monitoring results and retaining the ability to correct course. In other words, AI in research should be seen as an enhanced collaborator, not a replacement for expert judgment. As some industry practitioners put it, “AI should be a collaborator, not a conqueror: a tool that empowers researchers, not something that replaces them.” Keeping humans “in command” is also crucial for ensuring ethical principles are respected: only human sensitivity and judgment can fully assess implications and context, correcting direction if an algorithm suggests inappropriate actions. Finally, clearly defining responsibilities also means establishing protocols for algorithm auditing and incident management: if something goes wrong (e.g., an inadvertent privacy breach, a major data error), there must be an intervention and communication plan, with accountability toward stakeholders.
Algorithm transparency and explainability:
Closely linked to accountability is the explainability of decisions produced by AI. Ethically, individuals are considered to have the right to know how and why an algorithm produced a certain result—especially if it affects them or influences decisions. In market and social research, this implies that professionals should be able to explain to clients (and sometimes to participants) the general criteria by which AI analyzed data. For example, if a machine learning model segments consumers, the researcher should be able to describe the most influential variables and the basic logic behind the segmentation. Total opacity and algorithmic secrecy are difficult to reconcile with ethical transparency principles. ESOMAR guidelines explicitly encourage providers of AI solutions for research to “communicate the type of AI technologies used in the service” and, where applicable, to “clearly indicate when images or texts are generated by AI.” Even if technically complex, striving to provide understandable explanations of AI’s role increases client trust and supports independent evaluation of results. In some regulated contexts, transparency becomes mandatory: for example, some regulatory proposals require declaring the use of AI-generated content in public communications to avoid misleading people. For researchers, embracing transparency means documenting processes, communicating limitations and error margins of the AI used, and never presenting AI output as absolute truth, but as a decision-support tool to be interpreted.
Algorithmic discrimination:
A specific ethical risk related to bias is algorithmic discrimination—when an AI system unfairly disadvantages certain groups (often protected groups by ethnicity, gender, age, etc.). In opinion and market research, this could manifest if an algorithm systematically excludes responses from a minority group because they are “noisy,” or if a predictive model performs well on the majority population but makes poor estimates for subgroups. As noted, such disparities often originate from unbalanced data: if a group is underrepresented in training data, AI will tend to perform worse for that group, amplifying inequity. Identifying and preventing these forms of discrimination is an ethical imperative. This may mean collecting more balanced data, applying debiasing techniques, or auditing outputs to detect differential treatment. Organizations such as ESOMAR emphasize this point: because AI learns from historical data, it can reproduce societal prejudices and lead to discriminatory outcomes. For example, if certain communities responded less to online surveys in the past, a predictive model might systematically underestimate their future opinions, reinforcing a vicious cycle of exclusion. To counter this, beyond the technical solutions mentioned, it is crucial to embed ethical principles into AI design: internal guidelines, ethical checks during model development, and possibly algorithmic impact assessments focusing on fairness. In Europe, the upcoming AI Act classifies as high-risk those AI systems used in contexts that can harm fundamental rights through bias (for example, people-scoring systems) and introduces strict obligations to prevent discrimination. Even if market research typically does not fall under high-risk uses, voluntarily adopting strong non-discrimination standards is a sign of social and professional responsibility.
Existing guidelines and regulations
Given the ethical stakes and rapid technological evolution, institutions and professional associations have issued guidelines and regulations to steer the use of AI in research and protect participants. Here are some of the main ones:
- General Data Protection Regulation (GDPR): In force in the EU since 2018, the GDPR is the fundamental reference for any processing of personal data, including research that uses AI. Beyond requiring solid legal bases (informed consent, legitimate interest, etc.) to collect and use data, the GDPR imposes principles of transparency, data minimization, purpose limitation, and security that fully apply to AI projects. For researchers, this means limiting personal data collection to what is strictly necessary, providing clear privacy notices, and obtaining explicit consent for automated processing that goes beyond the participant’s initial expectations. Respecting data subjects’ rights is also crucial: for example, if AI is used to profile respondents (creating categories based on data), individuals have the right to access their data, correct them, or request deletion. The GDPR also establishes in general the right not to be subject to decisions based solely on automated processing that produce significant effects on the person (Art. 22)—a scenario that typically does not apply in market research, but that should be kept in mind if AI systems are developed that, for example, adapt questionnaires in real time by personalizing the user experience. In short, the GDPR provides a robust legal framework requiring researchers to implement AI in a way that protects privacy and participants’ rights, with severe penalties for violations.
- EU “AI Act”: The European Union is finalizing (with adoption expected in 2024 and application by 2026) a specific regulation on artificial intelligence, known as the AI Act. It is a horizontal legislative approach that classifies AI systems by risk level and regulates their development and use accordingly. The AI Act bans certain uses considered unacceptable (e.g., generalized social scoring, real-time facial recognition in public spaces, mass surveillance systems). For other systems, it introduces proportionate obligations: high-risk systems (not directly related to research, but for example AI in healthcare, education, employment) must meet strict requirements on transparency, robustness, documentation, risk management, and human oversight. Limited-risk systems must still comply with certain transparency standards (for example, informing users when they are interacting with AI) and conduct impact assessments. In the context of market and social research, the AI Act will serve as an overarching framework: although most uses (e.g., text analysis, survey chatbots) will likely fall into lower-risk categories, organizations will still need to ensure AI solution providers meet legal requirements (for example regarding training data quality, anti-bias measures, and auditing). It is worth noting that the AI Act strongly emphasizes principles such as transparency, accuracy, privacy, and algorithmic accountability, aligning with ethical standards the research sector has already begun to adopt. Once in force, this regulation will have global impact (as it will also apply to non-EU providers operating in Europe) and may become a model for similar regulations elsewhere. Research companies should already prepare by documenting their AI systems, training staff on AI compliance, and monitoring legislative developments.
- Professional codes and industry guidelines (ESOMAR, ESOMAR/GRBN, ASSIRM): Professional associations have often anticipated regulation by issuing ethical guidelines for AI use in research. ESOMAR, a leading international organization for market and social research, published together with GRBN a detailed “Guideline on Artificial Intelligence” (including, for example, a list of 20 questions to ask when evaluating AI service providers in research). These guidelines cover core aspects such as: the provider’s competence and experience in AI and research, explainability and reliability of proposed algorithms, adherence to ethical principles (absence of bias, fairness, beneficence toward participants), robust data governance (privacy, security, data ownership), and the presence of human oversight in processes. They encourage adopting explicit ethical principles in the development of AI solutions and mechanisms such as independent ethical reviews, participatory design involving diverse stakeholders (to capture socio-cultural implications), staff training in cultural sensitivity to avoid unintentional exclusion, and more. The ICC/ESOMAR International Code—an ethical cornerstone of the research profession—has also been updated to include references to new technologies, reaffirming that researchers must ensure transparency, honesty, confidentiality, and respect for participants even when using automated tools. In Italy, associations such as ASSIRM promote similar principles and support members in interpreting regulations like the GDPR in research contexts. In essence, the industry itself recognizes the importance of self-regulation: following these voluntary guidelines not only reduces the risk of ethical or legal harm, but also helps build trust among the public and clients, demonstrating that AI is used responsibly and professionally.
In conclusion, artificial intelligence applied to surveys represents an exciting frontier that is already putting powerful tools in researchers’ hands. At the same time, it requires new awareness of the limitations and responsibilities it entails. Algorithmic bias, lack of transparency, privacy risks, and ethical dilemmas require professionals to proceed with caution. As is often the case with disruptive technologies, the enabling factor is not replacing humans, but the quality of human–machine collaboration. In a sector built on participant trust and data reliability, AI must be guided by strong ethical principles and methodological rigor. Regulations such as the GDPR and the AI Act, together with ESOMAR guidelines and those of other associations, provide a valuable framework: legal and ethical compliance must advance hand in hand with innovation. By following these directives and maintaining a transparent and responsible approach, AI can truly enhance research not only in terms of efficiency, but also by raising its quality and ethics in the era of data and intelligent automation.
