Understanding Your Results

This guide helps you correctly interpret AudiAInce research results and avoid common misinterpretations.

Reading Result Summaries

Key Metrics

| Metric | What It Means | How to Use | | ---------------- | ------------------------- | ---------------------- | | Sample Size | Number of AI respondents | Larger = more reliable | | Confidence Level | Statistical certainty | 95%+ for key decisions | | Affinity Score | Overall product appeal | 60%+ is strong | | NPS | Recommendation likelihood | Compare to benchmarks |

Calibrated vs Raw

Raw results - Direct AI persona responses
Calibrated results - Adjusted using real-world base rates

Always use calibrated results for predictions. Raw results are useful for directional insights and comparisons.

Confidence Intervals

What They Mean

A result showing "65% (±5%)" means:

Best estimate: 65%
Range: 60-70%
Confidence: 95% (default)

How to Compare

Two results are significantly different if their confidence intervals don't overlap:

A: 65% (±5%) → 60-70%
B: 55% (±4%) → 51-59%
Conclusion: A is significantly higher (no overlap)

Two results are NOT significantly different if intervals overlap substantially:

A: 65% (±5%) → 60-70%
B: 62% (±5%) → 57-67%
Conclusion: Cannot conclude A > B (overlap)

Segment Analysis

Understanding Segments

Segments are groups of similar respondents identified by clustering. Each segment shows:

| Element | Description | | ------------ | ------------------------------ | | Size | % of sample in this segment | | Demographics | Common characteristics | | Affinity | Segment's product appeal score | | Motivations | What drives this segment |

Using Segments

Prioritize by size × affinity - Large, high-affinity segments are targets
Understand motivations - Tailor messaging to segment drivers
Note differences - What makes segments distinct?
Validate with interviews - Deep-dive on key segments

Segment Limitations

Segments are statistical groupings, not real people
Small segments (< 10%) may be unstable
Segment boundaries are fuzzy, not hard lines
Re-running may produce slightly different clusters

Question-Level Results

Response Distributions

Bar charts show response distributions:

Center of gravity indicates overall sentiment
Spread shows consensus vs division
Bimodal distributions suggest distinct groups

Interpreting Scales

For 5-point scales (1 = Strongly Disagree to 5 = Strongly Agree):

| Mean | Interpretation | | ------- | -------------------- | | 4.5+ | Very strong positive | | 4.0-4.4 | Positive | | 3.5-3.9 | Slightly positive | | 3.0-3.4 | Neutral | | 2.5-2.9 | Slightly negative | | < 2.5 | Negative |

Open-Ended Responses

AI-summarized themes show:

Common themes (frequency-ranked)
Representative quotes
Sentiment classification
Actionable insights

Comparative Results

A/B Comparisons

When comparing A vs B:

Preference % - Who would choose A over B
Attribute differences - Where A/B differ
Segment preferences - Some segments may prefer differently

Time Series

When tracking over time:

Look for trends (3+ consistent data points)
Note confidence intervals (changes may not be significant)
Consider external factors (seasonality, market changes)

Common Misinterpretations

"100% said they would buy"

Issue: Interest ≠ purchase. High interest is positive but conversion is lower.

Solution: Apply conversion benchmarks. 70% "definitely would buy" typically translates to ~15-20% actual purchase.

"Our product beat the competitor"

Issue: Statistical significance matters.

Solution: Check if confidence intervals overlap. A 52-48 split may not be meaningful.

"Segment X loves our product"

Issue: Small segments can show extreme results.

Solution: Check segment size. Segments under 10% may not be reliable.

"Results are wrong because..."

Issue: Results may be uncomfortable but accurate.

Solution: Consider if expectations were biased. Validate with other data sources.

When to Trust Results

High Confidence Situations

Large sample size (150+)
Calibration data available
Consistent with other research
Clear, specific questions
Well-defined product

Lower Confidence Situations

Small sample (< 100)
Novel product category
Broad targeting
Ambiguous questions
Limited product information

Taking Action

Strong Signals

Take action when:

Large effect size (> 10 percentage points)
No confidence interval overlap
Consistent across segments
Aligned with other data

Weak Signals

Investigate further when:

Small differences (< 5 points)
Overlapping confidence intervals
Segment disagreement
Conflicting with expectations

Statistical Methods - Understanding confidence
Calibration - How results are calibrated
Surveys - Running surveys effectively