Can Large Language Models Replicate Survey Data Across Topics?

“Synthetic samples” fail to replicate human response patterns on both political and non-political questions.

logo

In December 2025, Verasight asked members of the polling and market research communities to submit survey questions for us to ask a sample of 2,000 U.S. adults. From this, we created a LLM-generated dataset for those same human responses, and imputed their attitudinal responses given their demographic traits.

This report compares the response-by-question proportions in the human and LLM-generated survey data.

Read our full analysis here.

If you are planning on conducting a survey or are interested in collaborations on further synthetic research, contact us.

Key Takeaways

Mean absolute error by question category for single-answer questions. Lower values indicate better LLM performance.



Selection rates for multi-response question options. Points along the left edge represent options that human respondents selected but the LLM systematically ignored. The red region highlights options with >5% human selection but <1% LLM selection.



Comparison of LLM-imputed and human survey response proportions by category. Each point represents one response option for a single-answer question. The dashed line indicates perfect agreement.





1 Frequency of Digital Detox for Wellbeing

How often do you intentionally unplug from all digital devices (phone, TV, computer) for at least one hour to improve your wellbeing? (N = 1,000)
Response Real Synthetic
Daily 25% 0%
A few time per week 21% 14%
A few times per month 11% 53%
Rarely 26% 34%
Never 16% 0%
Other 0% 0%
I would rather not say 0% 0%

1.1 Crosstabs

Real Crosstabs

Age

Response 18-29 30-49 50-64 65+
Daily 22% 24% 26% 27%
A few time per week 18% 25% 21% 18%
A few times per month 15% 11% 11% 6%
Rarely 25% 28% 24% 28%
Never 18% 12% 17% 20%
Other 0% 0% 0% 0%
I would rather not say 1% 1% 0% 0%

Income

Response < $50,000 $50,000-$99,999 $100,000 - $149,999 > $150,000
Daily 26% 27% 22% 17%
A few time per week 23% 20% 19% 25%
A few times per month 10% 11% 13% 9%
Rarely 28% 24% 27% 34%
Never 13% 19% 18% 14%
Other 0% 0% 0% 0%
I would rather not say 0% 1% 1% 1%

Gender

Response Male Female
Daily 25% 25%
A few time per week 19% 23%
A few times per month 10% 11%
Rarely 27% 26%
Never 18% 14%
Other 0% 0%
I would rather not say 0% 1%

Education

Response HS or less Some college/2-yr degree 4-yr/post-graduate degree
Daily 25% 26% 24%
A few time per week 21% 20% 22%
A few times per month 9% 11% 12%
Rarely 28% 22% 28%
Never 17% 20% 13%
Other 0% 0% 0%
I would rather not say 0% 1% 0%

Party ID

Response Democrat Independent Republican
Daily 17% 29% 26%
A few time per week 22% 22% 19%
A few times per month 10% 10% 12%
Rarely 31% 25% 24%
Never 19% 14% 18%
Other 0% 0% 0%
I would rather not say 0% 1% 0%

Race/Ethnicity

Response White Black Hispanic Other
Daily 26% 23% 24% 23%
A few time per week 21% 23% 20% 23%
A few times per month 10% 17% 10% 6%
Rarely 25% 25% 29% 31%
Never 18% 12% 16% 15%
Other 0% 0% 0% 0%
I would rather not say 0% 0% 1% 1%

Synthetic Crosstabs

Age

Response 18-29 30-49 50-64 65+
Daily 0% 0% 0% 0%
A few time per week 21% 13% 12% 10%
A few times per month 34% 53% 58% 63%
Rarely 45% 34% 30% 26%
Never 0% 0% 0% 0%
Other 0% 0% 0% 0%
I would rather not say 0% 0% 0% 0%

Income

Response < $50,000 $50,000-$99,999 $100,000 - $149,999 > $150,000
Daily 0% 0% 0% 0%
A few time per week 3% 13% 20% 39%
A few times per month 41% 60% 60% 45%
Rarely 57% 27% 21% 15%
Never 0% 0% 0% 0%
Other 0% 0% 0% 0%
I would rather not say 0% 0% 0% 0%

Gender

Response Male Female
Daily 0% 0%
A few time per week 9% 19%
A few times per month 48% 57%
Rarely 43% 24%
Never 0% 0%
Other 0% 0%
I would rather not say 0% 0%

Education

Response HS or less Some college/2-yr degree 4-yr/post-graduate degree
Daily 0% 0% 0%
A few time per week 5% 10% 27%
A few times per month 42% 60% 59%
Rarely 53% 30% 14%
Never 0% 0% 0%
Other 0% 0% 0%
I would rather not say 0% 0% 0%

Party ID

Response Democrat Independent Republican
Daily 0% 0% 0%
A few time per week 31% 10% 5%
A few times per month 62% 50% 48%
Rarely 7% 41% 47%
Never 0% 0% 0%
Other 0% 0% 0%
I would rather not say 0% 0% 0%

Race/Ethnicity

Response White Black Hispanic Other
Daily 0% 0% 0% 0%
A few time per week 13% 15% 13% 17%
A few times per month 54% 57% 46% 55%
Rarely 33% 28% 41% 28%
Never 0% 0% 0% 0%
Other 0% 0% 0% 0%
I would rather not say 0% 0% 0% 0%

2 Importance of Undergraduate Degree in Today’s Job Market

How important is an undergraduate degree in the job market today? (N = 1,000)
Response Real Synthetic
Extremely important 20% 0%
Somewhat important 43% 100%
Not important or unimportant 23% 0%
Somewhat unimportant 10% 0%
Very unimportant 5% 0%

2.1 Crosstabs

Real Crosstabs

Age

Response 18-29 30-49 50-64 65+
Extremely important 29% 18% 20% 13%
Somewhat important 44% 40% 38% 50%
Not important or unimportant 18% 24% 28% 21%
Somewhat unimportant 3% 12% 10% 12%
Very unimportant 6% 5% 4% 4%

Income

Response < $50,000 $50,000-$99,999 $100,000 - $149,999 > $150,000
Extremely important 20% 16% 23% 31%
Somewhat important 39% 45% 40% 45%
Not important or unimportant 29% 23% 22% 10%
Somewhat unimportant 7% 12% 9% 9%
Very unimportant 6% 4% 5% 5%

Gender

Response Male Female
Extremely important 18% 21%
Somewhat important 41% 44%
Not important or unimportant 24% 22%
Somewhat unimportant 10% 9%
Very unimportant 6% 4%

Education

Response HS or less Some college/2-yr degree 4-yr/post-graduate degree
Extremely important 18% 15% 25%
Somewhat important 33% 44% 51%
Not important or unimportant 32% 23% 15%
Somewhat unimportant 11% 11% 8%
Very unimportant 6% 7% 2%

Party ID

Response Democrat Independent Republican
Extremely important 26% 18% 16%
Somewhat important 50% 39% 41%
Not important or unimportant 15% 30% 22%
Somewhat unimportant 5% 9% 14%
Very unimportant 4% 4% 6%

Race/Ethnicity

Response White Black Hispanic Other
Extremely important 16% 30% 21% 27%
Somewhat important 44% 36% 42% 43%
Not important or unimportant 23% 23% 25% 17%
Somewhat unimportant 12% 6% 6% 8%
Very unimportant 4% 5% 6% 5%

Synthetic Crosstabs

Age

Response 18-29 30-49 50-64 65+
Extremely important 0% 0% 0% 0%
Somewhat important 100% 100% 100% 100%
Not important or unimportant 0% 0% 0% 0%
Somewhat unimportant 0% 0% 0% 0%
Very unimportant 0% 0% 0% 0%

Income

Response < $50,000 $50,000-$99,999 $100,000 - $149,999 > $150,000
Extremely important 0% 0% 0% 0%
Somewhat important 100% 100% 100% 100%
Not important or unimportant 0% 0% 0% 0%
Somewhat unimportant 0% 0% 0% 0%
Very unimportant 0% 0% 0% 0%

Gender

Response Male Female
Extremely important 0% 0%
Somewhat important 100% 100%
Not important or unimportant 0% 0%
Somewhat unimportant 0% 0%
Very unimportant 0% 0%

Education

Response HS or less Some college/2-yr degree 4-yr/post-graduate degree
Extremely important 0% 0% 0%
Somewhat important 100% 100% 100%
Not important or unimportant 0% 0% 0%
Somewhat unimportant 0% 0% 0%
Very unimportant 0% 0% 0%

Party ID

Response Democrat Independent Republican
Extremely important 0% 0% 0%
Somewhat important 100% 100% 100%
Not important or unimportant 0% 0% 0%
Somewhat unimportant 0% 0% 0%
Very unimportant 0% 0% 0%

Race/Ethnicity

Response White Black Hispanic Other
Extremely important 0% 0% 0% 0%
Somewhat important 100% 100% 100% 100%
Not important or unimportant 0% 0% 0% 0%
Somewhat unimportant 0% 0% 0% 0%
Very unimportant 0% 0% 0% 0%

3 Ease of Access to Healthcare in the Last 6 Months

In the last 6 months, how often was it easy to get the healthcare, tests, or treatment you needed? (N = 1,000)
Response Real Synthetic
Always 43% 0%
Usually 35% 40%
Sometimes 15% 60%
Never 7% 0%

3.1 Crosstabs

Real Crosstabs

Age

Response 18-29 30-49 50-64 65+
Always 27% 36% 42% 67%
Usually 47% 37% 34% 24%
Sometimes 15% 20% 18% 7%
Never 11% 7% 6% 2%

Income

Response < $50,000 $50,000-$99,999 $100,000 - $149,999 > $150,000
Always 40% 40% 49% 50%
Usually 33% 37% 36% 34%
Sometimes 17% 17% 11% 10%
Never 10% 6% 4% 6%

Gender

Response Male Female
Always 48% 38%
Usually 33% 37%
Sometimes 13% 18%
Never 6% 7%

Education

Response HS or less Some college/2-yr degree 4-yr/post-graduate degree
Always 44% 40% 43%
Usually 31% 37% 38%
Sometimes 15% 16% 15%
Never 10% 6% 4%

Party ID

Response Democrat Independent Republican
Always 45% 36% 50%
Usually 38% 36% 31%
Sometimes 14% 18% 14%
Never 3% 10% 5%

Race/Ethnicity

Response White Black Hispanic Other
Always 47% 45% 32% 33%
Usually 33% 34% 41% 37%
Sometimes 14% 17% 18% 19%
Never 6% 4% 9% 11%

Synthetic Crosstabs

Age

Response 18-29 30-49 50-64 65+
Always 0% 0% 0% 0%
Usually 37% 32% 42% 53%
Sometimes 63% 68% 58% 47%
Never 0% 0% 0% 0%

Income

Response < $50,000 $50,000-$99,999 $100,000 - $149,999 > $150,000
Always 0% 0% 0% 0%
Usually 0% 35% 88% 96%
Sometimes 100% 65% 12% 4%
Never 0% 0% 0% 0%

Gender

Response Male Female
Always 0% 0%
Usually 48% 34%
Sometimes 52% 66%
Never 0% 0%

Education

Response HS or less Some college/2-yr degree 4-yr/post-graduate degree
Always 0% 0% 0%
Usually 17% 36% 67%
Sometimes 83% 64% 33%
Never 0% 0% 0%

Party ID

Response Democrat Independent Republican
Always 0% 0% 0%
Usually 43% 29% 53%
Sometimes 57% 71% 47%
Never 0% 0% 0%