Impact Analysis

Background

A longlasting task in basketball analytics has been to quantify player impact beyond the box score - beyond the amount of points, rebounds, and assists they racked up. It's obvious that a basketball player can do other things that impact winning which don't show up in the statsheet, and some of the things that do show up on the statsheet may not be all that important (i.e. a generally poor perimeter defender can average two steals per game because those steals may not outweigh the rest of their defensive possessions).

A traditional way to represent player impact without box score stats is to just use base plus-minus. For example, Steph Curry had a +4.0 plus-minus per 100 possessions in the 2021 season. This number means that with Curry on the court, the Warriors outscored their opponents by four points per every 100 possessions. You can also compare this to his net plus-minus compared to when he’s off the floor. Steph had a +8.6 on-off plus-minus, meaning that the Warriors outscore their opponents by 8.6 more points when Steph is on the floor than when he is not.

While easy to understand, the traditional plus-minus metric is very flawed. If an inferior player’s minutes heavily aligned with Curry’s, their plus-minus would look far better than it should just because they get to play with Curry. In other words, base plus-minus does not adjust for the strength of your teammates. Furthermore, while a quick look at on-off plus-minus may let you know that a player is carrying their team, it tells you more about how strong a team’s bench is than anything else. James Harden posted an on-off plus-minus of +9.1 in 2020, which pales in comparison with his +0.2 mark in 2021 (albeit on a small sample size). Did he get that much better? Of course not – the Brooklyn Nets are just far more equipped to play at a high level without Harden on the floor than the Rockets were.

So, we need to adjust for the other players on the floor. That’s the idea of Adjusted Plus-Minus (APM) – solving the system of linear equations representing the players on a court and the associating plus-minus for their duration on the floor.

Suppose that we have a matrix called \(A\) representing the players on the floor (one column for each player, a value of 1 if they’re on the floor for that stint and a value of 0 otherwise) and a vector \(b\) representing the plus-minus per 100 possessions for each stint. We can then solve for \(x\) which is a vector of coefficients corresponding to each players representing their on-court value.

$$Ax=b$$ $$A^TAx=A^Tb$$ $$(A^TA)^{-1}(A^TA)x=(A^TA)^{-1}A^Tb$$ $$x=(A^TA)^{-1}A^Tb$$

That’s the adjusted plus-minus solution. As you might anticipate, it has its own drawbacks. Most notably a high degree of variance. This problem can be alleviated with the addition of a filtering term that essentially acts as a penalty for outliers – it converges all values towards zero.

$$x=(A^TA+\lambda I)^{-1}A^Tb$$

An optimal \(\lambda\) value can be found which yields an approximate solution to the original \(Ax=b\) problem. This equation is the method used to calculate regularized adjusted plus-minus (RAPM).

Valorant

That was a lot of talk about basketball, but the same principles apply to Valorant. We can get a good sense of a player's quality by looking at some basic stats: kill death ratio, average combat score, etc. As we all know, though, it's possible to inflate these numbers without making winning plays. Baiting your teammates, getting non-impact frags at the end of rounds, etc. On the other hand, you can have a positive impact through plays that don't show up on the statsheet: strong util usage, game sense, leadership (making tactical decisions), etc.

We can try and look at match win percentage or round win percentage, but this can be affected by the players you play with (just like in basketball). Thus, we can calculate regularized adjusted plus-minus in a Valorant context.

The first step is data preparation, and the goal is to generate a "stints" dataset which will be used for computation. With 24 players to be analyzed, we generate 48 binary player columns - each player has a corresponding attack and defense column. These columns are the independent variables of the model represented in the matrix \(A\).

The dependent variable represented in the vector \(b\) is round differential per 100 rounds. So, let's say a five player combination played five games together, during which they had a 23-18 record over 41 attack rounds. That's a round differential of \(23-18=5\), which we standardize to a 100 round sample: \((5/41)\cdot100\approx12.195\).

Each row of our input data represents a single lineup's performance on one side (attack or defense). The five players in that lineup will have their attack or defense columns set to 1, thus the sum of the 48 binary player columns is always five.

Two other features are included in the model to improve fit: Map and Average Opponent Rank. So to be precise, each row in the input data represents a single lineup's performance on one side (attack or defense), on one map (ex: Haven), against teams with one average rank (ex: Ascendant). Yeah, it gets a bit complicated. But I found this to be necessary. Some maps are more biased towards attack or defense than others, so the model should be able to account for that. Furthermore, not considering the average opponent rank means there is absolutely no consideration for opponent strength at all. This penalizes higher ranked players (who will play better players) and inflates lower ranked players (who will play worse players) based on how Valorant matchmaking works. The average ranks are simplified - opposing teams with an exact average rank of "Diamond 1" and "Diamond 3" are both treated as "Diamond" in order to have a parsimonious model.

And that's essentially all there is to it. The one extra detail is that we keep track of rounds played in order to be used as a weight. After all, an extremely high round differential is more significant if it's maintained over a large sample. Afterwards, the actual ridge regression is run, spitting out 48 coefficients of interest (disregarding map and average rank coefficients) - each player has an "attack" and "defense" coefficient representing their Attack RAPM and Defense RAPM respectively. These values can be summed to obtain their Total RAPM.

Results

The following table shows the results with \(\lambda=100\). All players were included in the analysis, although only players with at least 1000 rounds played are actually shown below.

Great. And now for some quick caveats.

One criticism of RAPM is that its values have no intuitive meaning and the relatively arbitrary choice of \(\lambda\) affects this value differently for different players (fewer rounds means stronger regularization towards zero). While methods of hyperparameter selection exist, this part of the methodology is still something to consider.

Furthermore, no prior is used in this version of RAPM. While some modern NBA impact metrics like EPM and LEBRON are based on a box score and RAPM component in order to improve performance, NPI RAPM (non prior informed RAPM) is purely based on participation data.

Additionally, while RAPM traditionally takes both teammate and opponent strength into consideration by having all players as features in the model, this version does not because 99% of the time, opponents are faced for one game and one game only. This data does not come from a controlled league environment like the NBA - it's more akin to a single traveling team facing diverse competition. While opponent strength is somewhat considered by including a feature for the opponent team's average rank, it's not quite the same for a few reasons. One, rank fluctuates over time simply due to the phase of the game - at the start of every "Episode", players are assigned a lower rank so that they can work their way back up again. Also, it doesn't consider individual players. Is there a difference in quality between a team with two Immortals and two Silvers versus a team filled with fringe Platinum/Diamonds? Maybe, but we won't know that by taking a simple average of opponent rank.

Overall, RAPM is not viewed as an end-all-be-all in basketball and it definitely shouldn't be in Valorant as well. It's an interesting way to analyze the game a little differently, though, and can provide some nuance to player evaluation.