How Do Impact Scores in Experience Graphs work?
)
When analyzing insights and their impact on user experience, it's important to understand how impact scores are calculated and displayed in the experience graph.
This article explains the methodology behind the impact score calculations.
Why We Use Weighted Averages
Our system uses a weighted average approach to calculate impact scores rather than a simple sum. This method provides several benefits:
Comparability across steps
Normalization of scores
Flexibility in assigning importance to insights
Calculation Method
The impact score for a set of insights is calculated using a weighted average of individual insight scores. This means that:
Each insight contributes proportionally to the final score
The total number of insights doesn't skew the results
Insights can be assigned different weights based on their importance
Benefits of Averaging
Averaging impact scores offers several advantages:
Consistent Scale: Regardless of the number of insights, the final score remains on a consistent scale, typically between -1 and +1.
Fair Comparison: Steps with different numbers of insights can be compared more accurately.
Balanced Representation: Both positive and negative impacts are represented proportionally
Example Scenario
Consider a case where three insights have impact scores of -1, -1, and +1. While a simple sum would result in -1, the graph might show a value like -0.8. This discrepancy is due to the weighted average calculation.
Comparing Steps with Different Insight Counts
Consider this scenario:
)
Using averages, both steps show a similarly positive experience (score ~1). With simple sums, Step 1 would appear much more positive, potentially leading to misinterpretation.
Customizing Insight Importance
If certain insights are more crucial than others, you can adjust their weights in the calculation. This allows for a more nuanced representation of impact based on the relative importance of each insight. By understanding this methodology, you can better interpret the impact scores and graphs in your user experience analysis, leading to more accurate insights and informed decision-making.