Riot Games, the developer behind League of Legends (LoL), VALORANT, and Wild Rift, in collaboration with Amazon Web Services (AWS) introduced ‘Win Probability’ to the 2023 League of Legends World Championship broadcast. Analysing key data points throughout gameplay, the ‘Win Probability’ stat is designed to educate and inform fans as to how teams have fared when in similar competitive scenarios.
Riot Games recently began rolling out new machine learning-powered insights to boost the fan experience during esports broadcasts. Inspired by the in-depth statistics AWS and other global sports leagues offer during broadcasts, Riot collaborated with AWS to harness in-game data to make real-time predictions using Amazon SageMaker, a fully managed machine learning service. The first test of the ‘Win Probability’ stat was during the 2023 Mid-Season Invitational (MSI 2023) in May. The fan response to the stat was immediate and enthusiastic, encouraging the Riot team to deploy it in future esports events.
Working with the AWS Professional Services team, Riot created a self-serve machine learning workflow that Riot data scientists can use to create new esports stats. This lets the Riot team focus on data exploration, modelling, and experimentation, and rapidly deploy models securely into production-ready environments. Now, with a framework in place that scales insights accurately and reliably, Riot can deploy new stats in as few as six weeks, and apply the core principles of the workflow to other games.
Why introduce ‘Win Probability’ to LoL Esports?
To help fans better understand factors that influence a LoL Esports competition, Riot worked with AWS to develop a real-time stat that visualises how various aspects of the competition impact a team’s chances of winning. For accuracy, the model behind the ‘Win Probability’ stat generates a new prediction for every second of gameplay.
“With Win Probability, we’re able to visually tell the story of our sports in an exciting new way. Win Probability is a compelling narrative tool that can help fans realise the impact of certain in-game events, and how the course of any game can be decided in a few pivotal moments. By introducing Win Probability at Worlds, we hope it brings an innovative element to our fans’ experience, and enhances unforgettable moments like Gumayusi’s insane Baron steals in game 5 of the Worlds 2022 grand final,” said John Pham, Riot Games Win Probability Product Lead.
There are frequently big swings in a LoL match, and the ‘Win Probability’ stat helps contextualise key turning points for the viewer, while deepening their understanding of the game. The stat can also inform gameplay in hindsight, and help players understand why games unfolded as they did.
Following the initial test of the ‘Win Probability’ stat during MSI 2023, a viewer survey indicated that more than 63 percent recalled having seen the stat, with 94 percent finding it useful, and nearly 80 percent saying they liked or loved it. Here’s (30:31) a look at the stat’s initial test during MSI 2023 and how it helped the broadcast commentators underscore the importance of a particular series of events on the outcome of the match.
Developing the ‘Win Probability’ stat
Development of the ‘Win Probability’ stat started with data from LoL Live Stats, a source that continually streams data from every game in the tournament as it is broadcast. To train the model, Riot leveraged a variety of data points to track performance across each of the five individual players on two opposing teams, as well as overall team progress. This meant evaluating standard activity like the amount of gold accumulated (or “gold percentage”) per player/team and downtime while waiting to respawn after a player dies. This occurred alongside unique gameplay opportunities like the killing of an Elder Dragon, giving team members the advantage of a temporary superpower or “buff” and then, in some cases, comparing that data against players on the other team. A dozen gameplay data points were tracked for the initial deployment of the ‘Win Probability’ stat.
With LoL being continually revised with new patches, the team needed to ensure the data used for training the model was relevant to the current build. The AWS team worked with Riot to identify the appropriate patches, and collect data that would most impact the play of the current game, generate stats, label them, and test the impact of those features on win/loss.
The AWS team used correlation charts along with trial-and-error and the guidance of subject matter experts to assess the relevance of particular features, and built this functionality within the model for automated, consistent results going forward. Overall, the workflow was designed to be modular, giving Riot the flexibility to add, remove, and adjust pieces as needed.
A unique challenge for the project was determining how to create time-aware metrics in an offline manner to train the model, then generate those metrics in real-time. That required the creation of a platform and the offline feature store it enabled. In Amazon SageMaker, the AWS team created an online feature store to use the same elements for inference. Then, to display the data live during the broadcast, the AWS team assisted in the system build that makes the data accessible for use by Riot to generate win probability stat graphics.
To keep the Win Probability stat relevant as the game evolves, Riot continues to train the model with current data in the offline database. Additionally, Amazon SageMaker includes a tool to help identify and compensate for data drift that may impact predictions over time.
Unlike traditional sports metrics, which require pre-planning and physical sensors to capture data, nearly any esports game aspect can be tracked on-the-fly for future models and stat implementations. With the power to move quickly and react to player and fan interest, Riot can more easily create esports broadcast experiences that are engaging and help grow viewership. The opportunities are boundless as Riot and AWS continue to collaborate on new stats that can be deployed without any custom development in just a few weeks.