Science of Chess: Let your Brilliance be your guide

Improvers are often told to focus on learning from failure, but studying success might be more useful.

Some time ago, I was fortunate to be invited to be a guest on NM Ben Johnson's Perpetual Chess Podcast to talk about winning streaks in chess. Towards the end of our conversation, he asked me a question I still think about from time to time, and it's a question that's always worth asking yourself if you have aspirations of improving your game: "How serious are you about getting better?"

I only just now realized how similar our overall look was during this podcast. Maybe I should go back to some brightly-colored eyeglasses.

My answer involved a bit of hemming and hawwing, followed by vague appeals to the demands made on my time by my job, my everyday duties, and my love of watching stupid videos on the internet. Look, here's the thing: Like so many other things in life, I'd love to be better at playing chess than I am but thus far it turns out that I'm not so interested in this goal that I'm willing to carve out the time to actually work towards it with any real diligence. I have some idea what kinds of work I could be putting in to try and improve my play, but those activities are never as appealing as simply playing more games or spending my time reading books, drawing or painting something, or taking a walk.

But which activities actually do support chess improvement? If there is one piece of advice I think all improvers have heard, it's to devote more time to analyzing your play. While it's tempting to keep queuing up Bullet and Blitz games, I suspect nearly every chess coach you'll meet would tell you that working through your games with a careful and critical eye is simply non-negotiable. More specifically, the idea of learning and growing from studying one's failures is a common refrain. For example, I'm sure you've come across quotes like these that are intended to highlight the importance of treating mistakes as learning opportunities:

“When you lose, you have made a mistake, and that can help you learn - I never lose. I learn.” Tanitoluwa Adewumi (sourced in this NYT opinion piece: https://www.nytimes.com/2021/05/08/opinion/sunday/homeless-chess-champion-tani-adewumi.html)

"You may learn much more from a game you lose than a game you win. You will have to lose hundreds of games before becoming a good player." -José Raúl Capablanca (Irving Chernev's "The Chess Companion")

"We learn the most from our aches. The 2008 World Championship loss to Vishy was one of my greatest lessons.” — Vladimir Kramnik .

I had a hard time finding good verification for some of these.

There are tons of variations on this idea out there, filling many an inspirational chess list with hope that our blunders, our misses, and our inaccuracies might be the key to deeper mastery if only we're willing to look at them without shame. It seems sensible, right? Though I much prefer congratulating myself on the mating attack that was actually well-coordinated for once, working out what was lacking in the plans that failed is surely what I should be spending more time on.

Studying wins vs. losses - a regression analysis of ELO outcomes (Yiannakoulias, 2026)

The short article I want to tell you about here uses a straightforward statistical analysis to challenge this idea that we learn the most from failure. Specifically, the author chose to examine how three key factors might predict change in ELO over time: (1) How many games did you play?, (2) How many of your losing games did you analyze?, and (3) How many of your winning games did you analyze? The big idea here is that if failure really does teach us more than success, higher rates of analyzing losses should predict greater positive change in ELO over time.

There are two things about the data the author chose to work with that I quite liked. First, the data set in question only included Lichess players between 1600-1800 ELO. Compared to the frequent focus on GM (or other titled players') abilities, I thought this was a nice choice and likely means we're talking about players who are thinking about improvement a fair bit. Second, the author only investigated one month of Blitz play (3-5 minute time controls during May 2023). While I can see an argument for focusing on Rapid or Classical play instead, I think Blitz is useful for these purposes due to the increased incidence of mistakes during play (which we'd expect are opportunities for learning!). Seeing a one-month snapshot also struck me as a nice way to build in some easy potential for replication by repeating this analysis with other months and other sets of players (though this isn't done here).

Before we do anything more sophisticated, how did this cohort of ~2000 players fare in terms of ELO change after the nearly 2,000,000 games they played during the target month? Below, you can see a messy plot of all the time-series data for ELO change for each player in the data set: On average, nothing much happened, but there are players who got better, players who got worse, and lots of variation in between. This last point is important because the author wants to try and explain that variability using the predictor variables I mentioned above.

Figure 1 from YiannaKoulias (2026) - A superimposed plot of ELO fluctuations during the month of May 2023 for all ~2000 players in the data set used for the main analysis in this paper. Players tended to bounce around within a +/- 75 ELO band centered on their starting rating.

To achieve this, the authors used a regression model that included ELO change as the outcome we're trying to predict and the data describing rates of game play, win analysis and loss analysis. A nice subtlety that's worth pointing out here is that win percentage was also included in the model as a predictor variable. But why? Don't we already know that winning more games means your ELO will go up? In a word, Yes! We DO know that and including win percentage in the full model is a way of trying to remove the inevitable growth in ELO that's just due to winning more. The result is that any meaningful relationships we find between ELO change and the other predictors are less likely to just be artifacts of some players racking up more W's - the author has statistically controlled for that contribution.

Speaking of those other meaningful relationships, it turns out that there are a few worth talking about and the table below summarizes the key results. If you're not familiar with the numbers you tend to find in tables like these, the two things I'd suggest you look at to get a feel for what's happening are the p-value (labeled Pr(>|t) in the rightmost column) and the t-value in the 3rd column. To do some grave violence to how you interpret regression results, the p-value tells you how likely it is that the relationship between each predictor and the outcome variable (ELO change) happened by random chance. This means that a small value (a low probability that random chance is responsible) signals more confidence that there was a meaningful relationship between the predictor and the outcome. For our purposes, sign of the t-value (whether it is positive or negative) tells you if that relationship was itself positive or negative: A negative t-value means that a larger value for the predictor (say, more games played) led to a smaller value for the outcome (poorer change in ELO over time).

Table 2 from YiannaKoulias (2026) - Analyzing Wins positively predicts ELO change, but Analyzing Losses does not! Maybe it's not so valuable to study your mistakes.

With that in mind, there are two main points to take away from this table. First, let's start with our idea that studying your losses makes you better - learn from your failures! Let your blunders teach you the hard lessons! Unfortunately, it looks like there is little evidence that these aphorisms are true. That large p-value (it only gets as big as 1) means that we shouldn't have much confidence in the relationship between analyzing losses and ELO change (good or bad). Studying losses more didn't predict greater ELO growth.

But what about studying your wins? That's our 2nd row in the table and...well, things look a lot more interesting. The very small p-value and the large, positive t-value suggest that reviewing your victories likely does meaningfully predict a more positive ELO change. That is, study your brilliancies more often and it looks like you get better!

There are two interesting caveats however, the first of which has to do with those additional terms labeled 'wins2' and 'wins3' in the table. These are transformations of the "wins" data into higher-order polynomials (introducing a predictor that is the original "wins" data squared and cubed, respectively) that the author only included because the linear term was significant. Using the model the author estimated from this data, we can look at the predictions for what will happen to ELO over time given different rates of analyzing your wins, and this lets us see what these non-linear terms mean more easily.

Figure 2 from YiannaKoulias (2026) - Analyzing more of your wins is related to more positive ELO change, but only up to a point. Players who analyzed the largest fraction of their wins (the yellow 24% line above) had a rise-and-fall over time that isn't evident for players who analyzed fewer wins.

Briefly, analyzing your wins helps up to a point, but it looks like eventually the effect of doing so kind of saturates out. Players who analyzed 1 out of 8 wins (the blue line above) got more bang for their buck in the form of consistently increasing ELO as they played more. Players who analyzed twice as many of their wins (nearly 1 in 4, corresponding to the yellow line), however, exhibited a rise and fall indicating that this extra effort didn't translate into a higher rating.

The second caveat is a more important one about the methodology: The author did not have access to who requested the analysis, only that an analysis was requested. This means that it is possible (and likely) that there are instances in the data where the winning player did not request the analysis themselves. To quote the author's discussion of this point: "...some players will be characterized as having analyzed their game without having done so. Importantly, both of these issues bias our results to the null; that is, whatever effect we see is likely an underestimate of the true effect. This is because mischaracterizing players as having analyzed their game when they have in fact not analyzed their game adds statistical noise to the analysis variable that weakens associations between analysis and performance."

From my perspective, it would obviously be better to know who asked for the analysis, but I also get the argument the author is making here. Even if you are uneasy about this aspect of the methods, how do you explain the observed effects (and the observed null outcomes)? There are probably some ways to tell a different story (and if you have one, I'd be curious to hear it in the comments), but I think it's a little tricky. That said, this is also an interesting case where the nature of the available data limits what we know about behavior: Big databases make a lot of things possible, but also put you at the mercy of what's in those repositories. Still, I think there remains an intriguing implication from this analysis despite these limitations: Studying your successes may be more effective than studying your failures.

But why?

As I was just finishing this article and preparing to blog about it, I happened to see a banner ad on chess.com for their Advanced Stats tools that highlighted the option to look at all of your brilliant moves. In light of the results I had just finished taking in, I decided that reveling in my best moments over-the-board wasn't a terrible idea after all and dialed mine up.

What struck me about my own (rather paltry) collection of Brilliant moves is that there were a number of instances in which I definitely didn't see all the way to the reason that my Brilliant move was a good idea. Conversely, do you want to know something that's frequently the case about my blunders? I know EXACTLY why they were a mistake: I hung a piece, I missed a fork, or I completely ignored a pending mate. These kinds of mistakes happen in Blitz when you're at my level and while it's important to work towards making fewer of these mistakes, it's not like there's a deep concept I need to learn by reviewing them. To put it more bluntly, I always know that my Blunders are bad, but I don't always know that my Brilliant moves are good.

This paper isn't specifically concerned with single moves like Blunders and Brilliant moves, but I think the difference between analyzing wins and losses may work similarly. Being a fairly mid player who would like to get better, I can almost always tell you why I lost because the reasons are often embarrassingly simple (see my list of blunders above!). Oddly, however, I can't always tell you exactly how or why I won. Sure, sometimes it's down to the clock, or a consequence of my opponent making a conspicuous mistake of their own, but sometimes I'm as surprised as anybody that I ended up in a position that absolutely constricted my opponent: When did that happen? Which of my choices led to the superior position? How do I do that again? Some of the skills I need to develop have to do more with cultivating a win rather than avoiding a loss. Maybe that makes reviewing and remembering success critically important.

Obviously there isn't a single "Do this, not that" answer to how to improve or how to study. What I think is compelling about this paper, however, is that it uses some simple tools to suggest that there may be some gaps in the conventional wisdom about failure, success, and what we really need to learn to be better chess players. With that in mind, I'm going to take a look at the last absolutely smashing victory I had and make sure it wasn't a total accident.

Support Science of Chess posts!

Thanks as always for reading! If you're enjoying these Science of Chess posts and would like to send a small donation my way ($1-$5), you can visit my Ko-fi page here: https://ko-fi.com/bjbalas - Never expected, but always appreciated!

References

Yiannakoulias, Niko. (2026). Learning From Wins and Losses: An Analysis of Improvement in Online Speed Chess. Simulation & Gaming. 10.1177/10468781261443352.