Building an Item Recommender When Winning Explains Almost Nothing

Items matter in League of Legends. But if you try to build a model that learns which items matter by looking at whether the buyer won, you hit a wall fast. Win is a weak signal for item quality, and it took us several attempts to figure out why and what to do about it.

This article covers the problem, the failed approaches, the approach that works, and where it still falls short.

---

Why Win Is a Weak Signal for Items

Consider a single item purchase. You are one of ten players in the game. This item is one of roughly four completed items you'll buy across a 25-35 minute match. The game's outcome depends on champion matchups, mechanical skill, macro decisions, team coordination, draft, and whether your bot lane goes 0/5 before 10 minutes.

Items sit on top of all of that. A good item choice might move your win probability by a fraction of a percent. A bad one might not matter at all if your team is ahead by 5,000 gold. The purchase decision is a small input into a noisy, multi-player system, and the outcome (win/loss) is temporally far from the decision and influenced by dozens of other factors.

When you train a model to predict win from game state + item purchased, the model learns the obvious: gold leads predict wins. Level advantages predict wins. Kill participation predicts wins. The item itself gets ignored entirely.

This makes intuitive sense. If you showed someone a snapshot of a game at 15 minutes and asked "will this team win?", they'd look at gold, kills, objectives, and team comp. They wouldn't look at whether the top laner bought Sunfire Aegis or Hollow Radiance. The item choice matters, but in a prediction task it's completely drowned out by stronger signals.

---

What We Tried

V1: Pointwise Classification

The first attempt was straightforward. For every item purchase event in our dataset, build a row with the game state at that moment (gold, level, stats, team comp, lane matchup) plus the item that was bought. Target: did this player's team win?

Model: XGBoost classifier, ~125 features.

The model trained fine and predicted wins well. But when we checked whether it actually learned item preferences, the answer was no. Feature importance showed the model was routing almost entirely through gold differential and player stats. The item was invisible to the model.

Changing item_id in a prediction didn't move the output at all. Even after removing outsized features like gold differential, the model still found ways to ignore items. It was a win predictor that happened to have an item column. It wasn't an item recommender.

V2-V3: Isolating the Item Signal

We tried to rescue the pointwise approach. The idea: if the problem is that game state dominates the prediction, strip it out first. Train a baseline model that predicts win probability from game state alone (no item information). Compute residuals: what's left over that the game state can't explain. Then train a second model to predict those residuals from the item choice.

This is conceptually sound. It's a form of residualization, similar to what we use in our player insights system with Double ML. If items have any signal at all, isolating it from the dominant game-state signal should surface it.

In practice, the residuals were too noisy. The unexplained variance in win/loss after removing game state is large (games are inherently unpredictable), and items explain only a tiny fraction of what's left. The signal-to-noise ratio was still too low to learn stable item preferences.

We also explored using player performance metrics (damage, KDA, objective participation) instead of win/loss as the target. This narrows from 10 players to 1 and gets temporally closer to the item purchase. It didn't help. Items are a small contributor to individual performance too, and the model still ignored them.

---

V4: Pairwise Ranking

What worked was reframing the question.

Instead of asking "does buying this item lead to winning?", ask: "given this game state, is item A a better choice than item B?"

This is a learning-to-rank approach. The model doesn't predict absolute outcomes. It predicts relative preferences between items, conditioned on the full game state. The target shifts from a distant, noisy binary (win/loss) to a tighter comparison: which item did winning players actually choose when both were options?

How It Works

The model takes a snapshot of the game at the moment you're about to buy, and compares candidate items in pairs. For each pair, it asks: "given everything happening in this game right now, which of these two items is the better purchase?" Every candidate gets compared against every other candidate, and the final ranking comes from averaging those pairwise scores.

Each comparison is fed roughly ~300 features, which break down into two groups: what's happening in the game (the context) and what's different between the two items (the comparison).

The prior diff is the baseline. It encodes what's historically good for this champion/role/slot without any game context. The stat diffs tell the model what is different between the two items. The game context tells it whether that difference matters right now. 30 more armor than the alternative is worth a lot against a 4-AD team. It's worth less when the enemy is mostly magic damage.

The item indicators let the model learn item-specific preferences that aren't captured by raw stats (passive effects, unique interactions, build path synergies).

Why Pairwise Fixes the Signal Problem

The core issue with V1 is that the model gets no loss pressure to learn items. When the target is "did you win?", items contribute so little that even after removing dominant features like gold differential, the model finds other signals that predict wins better. There is always something more predictive. Ignoring items never costs enough loss to matter.

Pairwise ranking fixes this. Both rows in a pair share the same game state. The only difference is the item features. If the model ignores items, it's guessing 50/50 and eating maximum loss. There's nowhere else for the gradient to go.

Priors

For every champion/role/build slot, we compute a prior score for each item from historical win rates and pick rates:

The prior encodes what works on average across all games. The difference between the two candidate items' prior scores (prior_diff) is fed into the model as an input feature alongside the ~300 game state features.

The model only reduces loss by learning when the game state justifies deviating from that baseline. If an item's prior is lower but the stat diffs and enemy composition make it the better choice right now, the model has to learn that.

Training Data

For every item purchase in a winning game, we take the game state snapshot from the frame just before the purchase. Then we pair the chosen item against the other candidates for that champion/role/slot:

• (chosen, alternative): label = 1
• (alternative, chosen): label = 0 (flipped)

Each purchase generates multiple training pairs. Across millions of matches, this produced roughly 600M pairs per role (3B total across all five roles).

Inference

At prediction time:

1. Load the candidate items for this champion/role/build slot (top ~10 by prior).
2. Filter out items you already own, boots if you have tier-2, etc.
3. Build all pairwise combinations among the remaining candidates.
4. Score each pair: P(item A > item B | game state).
5. Average each item's scores across all its comparisons.
6. Return the ranked list.

---

Adapting to Enemy Composition

Most of the time, the model's deviations from the prior are small: boots recommendations shifting, a defensive item moving up or down a slot. The extreme cases are where the difference is most obvious.

Here's Ornn top buying his second completed item in two different games.

In the first game, the enemy team is 87% physical damage with a fed Miss Fortune. The model stacks armor: Sunfire Aegis #1, Plated Steelcaps #5. Kaenic Rookern, the premier MR item, drops to dead last. In a normal game you'd never see a second armor item recommended this early.

In the second game, the enemy team is 89% magic damage with a fed Sylas. The recommendations flip completely. Kaenic Rookern jumps to #1, Mercury's Treads rises to #4, and Sunfire Aegis falls to #7. Again, doubling up on MR this early only happens when the game state is this skewed.

Same champion, same role, same build slot. Completely different recommendations because the game state demands it. In a balanced 50/50 damage game, neither of these builds would appear. The model picks up on enemy damage ratios, the most-fed enemy's champion and gold lead, and team-level armor/MR averages to shift its rankings.

This kind of adaptation also shows up in other ways: recommending defensive items earlier when a specific enemy carry is ahead, favoring items you already have components for, and shifting AP/AD recommendations based on what your team is lacking.

---

Limitations

Candidate pool. The model only compares items that make it into the candidate list (top ~10 by prior for that champion/role/slot). If an item is rarely built on a champion, it may not enter the pool at all and can't be recommended regardless of how good the stat diffs look. The model can adapt within the candidate pool, but it can't consider items it's never seen players build in that slot.

Off-meta builds. AP Varus, lethality Caitlyn, support Senna mid. The training data is sparse for unusual champion/role combinations. Recommendations will lean toward the conventional build because that's where the pairwise comparisons are densest. The stat diff features can still do reasonable work here, but the item indicators and priors have less to go on.

First items. Early game states are similar across games, so first-item recommendations track priors closely. The model adds the most value on second, third, and fourth items where game states have diverged enough to matter.

Patch lag. After a patch that changes items or champions significantly, recommendations lag until enough new match data comes through. Priors update faster than the full model retrains.

Correlation, not causation. The model learns what winning players bought in a given game state, not what caused them to win. A winner choosing item A over B doesn't prove A was the better choice. Across billions of pairs the noise averages out enough to produce useful rankings, and conditioning on game state controls for most confounders, but this is a ranking heuristic, not a counterfactual. The main artifact is boots getting slightly underranked, since winners tend to have more gold and favor bigger items. It's still strictly more useful than a static tier list that ignores your game entirely.