Stop Interpreting Predictive Models as Causal: How We Built Player Insights

Machine learning models are good at prediction. Interpretability tools like SHAP are good at explaining what those predictions depend on. But when the goal shifts from predicting an outcome to deciding what to change to improve it, the gap between correlation and causation becomes dangerous.

This article walks through why predictive models mislead when used for causal reasoning, using a concrete business example. Then we show how this applies directly to League of Legends player insights, and what we do about it.

The core idea is simple: SHAP makes transparent the correlations picked up by predictive ML models. But making correlations transparent does not make them causal.

Prediction vs. Intervention

A prediction task asks: given what we know about this person, what outcome do we expect?

A causal task asks: if we change something about this person's situation, how does the outcome change?

These sound similar. They require fundamentally different assumptions.

A predictive model learns patterns from historical data. It assumes the future looks like the past. It captures correlations between features and outcomes. This is appropriate when you want to forecast: which customers will churn, which patients are at risk, which team will win.

A causal model asks a counterfactual question: what would have happened if we had intervened? If we had given this customer a discount, would they have stayed? If this player had placed more wards, would they have climbed?

The problem arises when people train a predictive model, interpret its feature importances, and then use those importances to make decisions. That is implicitly treating correlations as causes.

The Business Example: Subscriber Retention

To make this concrete, consider a company trying to reduce subscriber churn. This example is adapted from the SHAP documentation, which uses a synthetic dataset where the true causal relationships are known by construction.

The company collects data on 10,000 subscribers with eight features:

They train an XGBoost model to predict which subscribers will renew. The model performs well. They compute SHAP values to understand which features matter most:

Two results are immediately surprising:

• Bugs Reported has a positive SHAP direction: subscribers who report more bugs are predicted to renew at higher rates. Taken causally, this would imply we should ship more bugs to increase retention.

• Discount has a negative SHAP direction: subscribers who received larger discounts are predicted to renew at lower rates. Taken causally, this would imply giving a discount causes subscribers to renew less.

This is a recurring cognitive trap. We all know prediction and causation are different, but when a pattern sounds plausible we often treat it as if it were causal and move on. We usually catch the mistake only when the implied intervention is obviously absurd (for example, "ship more bugs"), so causal reasoning has to stay explicit even when the story passes a quick sniff test.

The True Causal Structure

This dataset was generated from a known causal structure. Here is the true causal graph:

The key structural facts:

• Product Need is an unobserved (unmeasured) variable representing how much a subscriber genuinely needs the product. It directly causes three things: subscribers who need the product more (1) report more bugs because they use it more, (2) tend to renew regardless of marketing efforts, and (3) receive smaller discounts because the sales team focuses discounts on reluctant subscribers.

• Monthly Usage and Last Upgrade are observed variables that directly affect Renewal and jointly influence the marketing team's Discount targeting.

• Ad Spending has zero direct causal effect on Renewal. It appears important only because Monthly Usage and Last Upgrade drive both Ad Spending decisions and Renewal.

• Sales Calls causally affect Interactions, which in turn causally affect Renewal. The true causal chain is Sales Calls → Interactions → Renewal.

• Economy directly affects Renewal with no confounding. It is the simplest case.

Understanding why the predictive model gets the wrong causal story requires understanding three different types of confounding.

Three Ways Predictive Models Mislead

The dataset has three structural problems that cause SHAP to disagree with the true causal effects. Each has a different shape and a different fix.

Bugs Reported has zero direct causal effect on Renewal. But subscribers who depend on the product report more bugs and renew at higher rates. The model sees the correlation and credits Bugs Reported. The same logic applies to Discount in reverse: the sales team targets reluctant subscribers with bigger discounts, so the model sees discounts as negative, even though discounts have a small positive causal effect.

When the confounder is unmeasured, no amount of clever modeling on observational data can recover the true causal effect. The only fixes are randomized experiments or instrumental variables.

Ad Spending has zero direct causal effect on Renewal. The marketing team targets ads at active users who recently upgraded, the same users who naturally renew. Ad Spending appears predictive only because it proxies for Monthly Usage and Last Upgrade. Because these confounders are observed, we can correct for this using Double ML (explained below).

Sales Calls genuinely cause higher Renewal, but only through Interactions. The predictive model splits credit between them, underestimating the total impact of Sales Calls. The fix is to either remove the mediator from the model or apply Double ML without controlling for Interactions (since it's a mediator, not a confounder).

How Double ML Works

Double ML isolates the causal effect of a treatment (e.g., Ad Spending) on an outcome (Renewal) by removing the influence of confounders from both sides.

Both models learn to predict their target from confounders. The residuals, the parts the confounders can't explain, represent "pure" variation in Ad Spending and Renewal. If these residuals aren't correlated, Ad Spending has no causal effect beyond what the confounders already explain.

For features that are causally independent of other features (like Economy), the predictive model's feature importance does reflect the true causal effect. No correction is needed.

For features with observed confounders, Double ML provides a practical correction. For features with unobserved confounders, observational data alone cannot recover the true effect.

How This Applies to League of Legends

When we train a model to predict rank from player behavior, SHAP tells us what correlates with climbing. But when we want to give players advice ("do more of X to climb"), we need causal effects. Our dataset has all three problems:

• Unobserved confounding. CS at 10 minutes strongly predicts rank, but the hidden confounder is mechanical talent. Players with better hands CS better and climb higher. CS is partly a symptom of skill, not purely a cause of rank. The same applies to KDA, damage per minute, and other raw mechanical stats.

• Observed confounding. Champion pool size predicts rank, but both are driven by experience (games played, months active). After controlling for experience with Double ML, the causal effect of pool size shrinks significantly.

• Redundancy. Wards Placed and Vision Quality both predict rank, but placing wards causes higher vision quality, which causes better outcomes. The model splits credit between them, underestimating the total impact of the actionable behavior.

Our Implementation: Double ML for Player Insights

We use Microsoft's EconML library to estimate causal effects across 37 player behavior features using Double ML (specifically, LinearDML with XGBoost nuisance models).

For each feature, we treat it as the "treatment" and use all other features as potential confounders. This estimates the Average Treatment Effect (ATE) on rank:

The results diverge significantly from naive feature importance:

The story changes meaningfully:

• Lane and execution signals are strongest. cs_at_10_avg (+1.62), ward_takedowns_vs_champ (+1.43), and cc_time_vs_champ (+1.12) are the largest positive ATEs.

• Communication and tempo remain high-impact. pings_per_minute (+0.74) and boots_timing_ratio (+0.78) are both strong positive effects.

• Negative effects concentrate in volatility/tilt patterns. surrender_rate (-0.45), retreat_vs_engage_ratio (-0.40), and consumables_per_game (-0.37) are the strongest negatives.

• Vision and consistency still matter. control_wards_per_game remains clearly positive (+0.41), and sticking_with_losing_champ remains clearly negative (-0.29).

• Build/rune optimality are positive but lower-leverage. They remain useful, but they are not among the top-magnitude effects.

Personalized Effects: CATE

Beyond average effects, we estimate Conditional Average Treatment Effects (CATE): how the causal effect varies depending on the player's profile.

This means the advice is personalized. A Silver player who never buys control wards has enormous headroom: the causal effect of going from zero wards to a few per game is large. A Platinum player who already averages three control wards per game sees a much smaller marginal effect from buying a fourth. Conversely, a Platinum player with a scattered champion pool might benefit heavily from focusing their picks, while a Silver player's champion pool barely matters because their fundamentals are the bottleneck.

This is what powers the player insights page: for each feature where a player underperforms relative to their rank, we estimate their specific causal effect and translate it to approximate LP impact.

Limitations

Double ML assumes that confounding is observed: that all relevant common causes are measured in our feature set. For features that are genuinely confounded by unmeasured variables (like raw mechanical talent), even Double ML may not fully recover the true causal effect.

We address this in two ways:

1. Feature selection. Some features are proxies for the unobserved confounder itself. Raw KDA, damage per minute, and CS differentials are essentially measuring mechanical talent with extra steps. They are to our model what Bugs Reported was to the subscriber model: driven almost entirely by an unobserved variable (talent / product need) that also drives the outcome. Double ML can't fix this because the confounder isn't in the dataset. So we exclude these proxies and focus the model on decision features: build choices, rune selection, ward placement, champion pool strategy. These are behaviors a player can change at their current skill level, making the causal estimates more likely to reflect actionable effects.

2. Confidence intervals. Every causal estimate comes with a confidence interval. Features where the interval includes zero are flagged as uncertain rather than presented as definitive advice.

3. Context over intuition. Sometimes the causal estimate contradicts conventional wisdom, and that's fine. We ship the estimate and give players the context to understand why, rather than filtering out results that feel uncomfortable.

The goal is not perfection. It is to be less wrong than naive feature importance, and to be honest about what we know and what we don't.

Conclusion

Predictive models answer: "What predicts the outcome?" Causal models answer: "What should you change?" These are different questions requiring different methods.

When you train a predictive model and interpret its feature importances as causal recommendations, you risk:

1. Confusing symptoms with causes. CS at 10 minutes correlates with rank, but part of that signal is underlying mechanical talent and context. Treating the full correlation as causal overstates intervention impact.

2. Mistaking confounded correlations for effects. High-level aggregates can look powerful in prediction while having much smaller direct intervention effects after adjustment.

3. Splitting causal credit across redundant features. Wards placed and vision quality share predictive importance, obscuring the true actionable behavior.

Double ML provides a principled correction for observed confounding. For unobserved confounding, only experiments or domain knowledge can help.

Our player insights system uses Double ML to estimate causal effects, then personalizes those estimates per player using CATE. The result is advice that is more likely to be actionable: not "players who do X tend to be higher ranked" but "if you changed X, here's our best estimate of how it would affect your rank."

Making correlations transparent is important. Making the distinction between correlation and causation transparent is essential.

The subscriber retention example and causal framework in this article are copied from "Be careful when interpreting predictive models in search of causal insights" by Scott Lundberg et al. in the SHAP documentation. Their original covers the statistical foundations in more depth and rigor than we do here. We adapted it for a League of Legends audience to explain how we built our player insights system. Go read theirs.