Managing technological risk in the AI era: ASSIRM report and impact on survey technology

The integration of Artificial Intelligence (AI) into market research and data collection has moved beyond mere conceptual experimentation. Today, we face a structured methodological and technological transition. The sector is evolving from an initial, uncontrolled enthusiasm for automation toward a more mature and critical awareness focused on data quality, process transparency, and technological risk management.

To trace the state of the art in the sector, an interactive assessment quiz report—curated by Fieldgood for ASSIRM—was released during the Market Research Forum (MRF26), involving 93 industry professionals. The results outline a precise map of current skills, highlighting both consolidated strengths, such as data protection and GDPR compliance, and areas of greater methodological vulnerability, such as understanding bias and algorithmic transparency.

As developers of survey software and data collection solutions, we analyze these data to understand how the adoption of AI is redefining the requirements of quantitative and qualitative market research, and what challenges await future technological architectures.

The big picture: awareness and the gap between supply and demand

The quantitative data from the ASSIRM report show a high overall level of awareness, but one with marked asymmetries. The overall average score obtained by participants was 3.8 out of 5, with 45% of respondents reaching the maximum score (Future Ready profile).

However, a breakdown of the sample reveals a deep divergence between those who produce research and those who purchase and use it within organizations:

• Research suppliers (66% of sample, 61 participants): Recorded an average score of 4.2 out of 5.
• Research buyers (34% of sample, 32 participants): Recorded an average score of 3.1 out of 5.

This 1.1-point difference on a scale of 1 to 5 demonstrates that the issue of technological risk linked to AI is currently far more addressed on the supply side (suppliers and software houses) than on the demand side (client companies).

For those working with survey platforms, this gap shows how crucial it is to be able to validate and justify AI output, offering clients maximum clarity and soundness in interpreting collected data.

The skills map: where we excel and where we risk

The analysis of the 5 thematic areas of the ASSIRM quiz draws a clear line between maturity in legal/compliance matters and mastery of actual algorithmic metrics:

• Data protection (88% correct answers – Q5): represents the highest point of correctness. Awareness of safe data usage and data leak prevention is now widespread.
• Governance and GDPR (83% correct answers – Q3): knowledge of European regulatory frameworks related to privacy management is widely covered.
• Human in the Loop (76% correct answers – Q4): good understanding of the need to keep the researcher at the center of the validation process.
• Data bias (69% correct answers – Q2): an area to reinforce; about 1 in 3 participants shows uncertainty regarding model-induced data distortion dynamics.
• Algorithmic transparency (68% correct answers – Q1): the lowest score of the survey, confirming that the inner workings of algorithmic “black boxes” are the least mastered concept.

Technical analysis: methodological risk of biases and the “black box” effect

The fact that algorithmic transparency (68%) and data bias (69%) were the least intuitive and least correctly identified answers is a significant signal. The most complex risks to intercept in market research are not regulatory or bureaucratic, but methodological and statistical.

Algorithmic Transparency: When using Large Language Models (LLMs) or machine learning algorithms for tasks such as automated open-ended response coding, sentiment analysis, or qualitative transcript synthesis, we deal with complex architectures. To guarantee the scientific reproducibility of the survey, a careful analysis and validation phase of the output is necessary before considering it final.

Data Bias: Generative models are trained on pre-existing data corpora that carry cultural, gender, linguistic, and socioeconomic biases. When AI is applied to dataset cleaning or missing data imputation, it risks introducing systematic biases that alter sample distribution and invalidate statistical significance tests.

Industry Profiles: the 5 archetypes of the AI-era researcher

Based on the correct answers provided in the quiz, the ASSIRM report profiled participating professionals into 5 archetypes describing the industry’s degree of readiness:

• Future Ready (45% – 42 participants): Respondents who scored 5/5 correct answers. They deeply understand the integration between technology and human control, basing operations on the principle that quality remains an indisputably human responsibility.
• Risk Manager (23% – 21 participants): Respondents with 4/5 correct answers. They show high oversight and a structured capability to manage technological issues.
• Quality Guardian (13% – 12 participants): Respondents with 3/5 correct answers. Focused on defending traditional quality standards, with room for improvement in understanding new algorithmic architectures.
• AI Aware (12% – 11 participants): Respondents with 2/5 correct answers. They possess theoretical knowledge of the phenomenon but require technical training to strengthen their skills.
• AI Explorer (8% – 7 participants): Respondents with 0–1/5 correct answers. They lack the minimum safety safeguards essential for conscious technology adoption.

Overall, 2 out of 3 participants (68%) fall into the top categories (Future Ready or Risk Manager). This demonstrates that the market research industry possesses a solid starting point and strong methodological maturity.

Operational priorities: efficiency vs. human centrality (Human in the Loop)

The qualitative section of the report, compiled from the open-ended responses of 59 participants (63% of the total), explores the evolution of practical priorities in daily AI use.

Key themes from qualitative analysis

• Efficiency and productivity (37% of citations): reduced execution times, automation of repetitive tasks, and operational cost savings represent the primary benefit sought by professionals.
• Human centrality / Human in the Loop (31% of citations): AI is strictly viewed as a support tool and never as a replacement for the researcher’s analytical intelligence.
• Reliability and data quality (22% of citations): the need to ensure verifiable, traceable data free from algorithmic hallucinations.
• Ethics, transparency and rules (22% of citations): the need to define clear ethical guidelines, strictly comply with GDPR, and protect respondents.
• Industry maturity and moving beyond hype (15% of citations): awareness of the need to scale back unrealistic AI promises in favor of methodological pragmatism.
• Synthetic data and personas (12% of citations): strong concern regarding risks associated with the improper use of artificial respondents instead of real consumers.

Market voices: the co-pilot balance

Direct testimonials collected from participants summarize the climate of cautious optimism characterizing the current phase:

“AI has become an indispensable aid, like a bright intern who still needs supervision.” (Future Ready respondent)

This analogy accurately reflects the Human-in-the-Loop (HITL) paradigm. Generative AI possesses extraordinary processing and execution speed but lacks contextual understanding, semantic sensitivity, and critical intelligence. Entrusting tasks to it without continuous supervision is equivalent to handing the scientific direction of a study over to an inexperienced intern.

“It’s no longer enough for AI to produce fast output; it must be verifiable and traceable. The question is not ‘how fast is it,’ but ‘how much can I trust it’.” (Future Ready respondent)

“AI is a facilitator; it shouldn’t replace the researcher, who stays behind the scenes in the design phase.” (Risk Manager respondent)

The synthetic data issue: the illusion of speed and risk of flattening

One of the most significant discussion points from the survey concerns the use of synthetic data and artificial personas. Although some vendors propose simulating respondents via LLMs as a cost-effective alternative to field research, industry professionals express deep reservations:

“To save money, people think about replacing real people with synthetic profiles that don’t reflect reality at all and flatten the results.” (Future Ready respondent)

From a data engineering and survey methodology standpoint, uncontrolled reliance on synthetic data involves three structural risks:

1. Elimination of variance and outliers: LLMs are designed to generate text by calculating the statistically most probable sequence. Consequently, simulated responses tend to converge toward average behaviors, erasing niche opinions, unexpected emotional reactions, and weak signals that represent the true strategic value of primary market research.
2. Risk of “echo chamber” or algorithmic resonance: if a company aligns its positioning or product development based on responses from synthetic models trained on historical web data, it is not measuring true consumer purchasing intent, but simply querying a statistical mirror of the past.
3. Flattening of sample representativeness: synthetic profiles suffer from hallucinations and training bias inherent in source models. Simulating interviews with specific sociodemographic segments (e.g., Generation Z or highly specialized B2B markets) via engineered prompts often leads to caricatured or stereotyped representations lacking scientific validity.

Implications for survey technology: how survey platforms must evolve

For software developers and providers of market research infrastructure (CATI, CAWI, CAPI), analyzing the ASSIRM report is not merely a snapshot of current industry status, but a roadmap for code design and user experience.

If industry priority has shifted from pure speed to traceability, reliability, and governance, survey technology must adapt across five core pillars:

• Conscious AI management in research: using AI for automations like open coding or questionnaire translation requires strict quality control. Overcoming “black box” limitations means establishing a responsible workflow where every algorithm-generated result is analyzed, contextualized, and verified to ensure coherence and validity.
• Natively Human-in-the-Loop (HITL) architectures: software should never apply irreversible changes to datasets autonomously. User interfaces must feature review panels (“Human Validation Dashboards”) where analysts can view AI interpretations, verify deviations, and approve, modify, or reject proposed coding before it becomes final statistical data.
• Data sovereignty and privacy-by-design architectures: the 88% accuracy rate on data protection confirms that privacy is an uncompromisable baseline. Software providers must ensure LLM integration or automated analysis occurs exclusively via isolated, private, GDPR-compliant instances. Survey data must not be used to retrain public models, preserving full control over personal (PII) and corporate data.
• Validation procedures and anomaly detection: controlling bias and quality in surveys requires thorough methodological verification of collected data. Research teams must apply precise audit criteria to identify suspicious response patterns, isolate improper submissions (including automated contributions from respondents), and monitor distribution stability during data cleaning.
• Clear separation between real data collection and synthetic modeling: survey platforms must preserve the purity of direct data collection from human consumers. Generative AI should primarily optimize preparatory phases (e.g., drafting initial questionnaire concepts) or facilitate post-fieldwork analysis, preventing synthetic simulation tools from being mistaken for primary survey data.

Conclusions and outlook for the sector

The report published by ASSIRM demonstrates that the market research community in Italy is approaching the AI revolution with cautious and responsible optimism.

The sector is fully aware of the efficiency potential offered by technology (37% of citations), but firmly rejects methodological shortcuts that compromise human validation (31%) or transparency toward clients.

For software houses developing technology for researchers, this evolution marks the end of blind automation. The role of modern survey software is not to fully automate research and eliminate human involvement, but to empower researchers with powerful, transparent, and secure tools that amplify analytical capability while maintaining rigorous standards in ethics, regulatory compliance, and methodological accuracy.

Original source: report on the interactive quiz results “AI in Market Research: Are you ready for technological risk management?”, curated by Fieldgood for ASSIRM on the occasion of the Market Research Forum (MRF26).