Another Look at the Chess.com vs. USCF Ratings Debate

Sort:
Jcbutler

There appear to be two kinds of people at Chess.com: those who care about their ratings and wonder how well they correspond to “real” ratings like USCF ratings, and those who dismiss the importance of ratings and question how they could possible mean anything at all. Another point of contention has been whether Chess.com ratings are inflated or "deflated" relative to USCF ratings. Fully aware of the risks of entering into a debate that has become heated at times, I humbly offer my own insights below, along with selected summaries from people who have considered this question before.


A couple of years ago, for example, DrawMaster examined data from more than 100 Chess.com members, rated 1500 to 1699, who also reported USCF ratings on their profiles. His main finding was that these members had Chess.com blitz ratings that were, on average, 73 points lower than their reported USCF ratings (1592 vs. 1665). He also provided a graph showing a linear relationship between USCF ratings and Chess.com ratings. He concluded tentatively that online blitz ratings "generally, if only slightly" underestimate OTB playing strength.

 

AdamRinkleff also argued that Chess.com blitz ratings were generally lower than USCF ratings, but he stated that the discrepancy was quite a bit larger. According to his own observations of about 20 people who maintain active ratings in both systems, AdamRinkleff suggested that Chess.com ratings are consistently 200-300 points lower than USCF ratings. Unfortunately, this claim was put forth from a small sample and without supporting statistical analysis. This led to a substantial debate on the forum and no real consensus. Many posters claimed that the two sets of ratings were like “apples and oranges” and could not be meaningfully compared.


In an attempt to test AdamRinkleff's hypothesis, ShindouHikaru posted data from 54 Chess.com members who had USCF ratings that had been verified by official records. Unlike DrawMaster's data, this sample was well above average in skill level, with many of the players taken from a list of titled players on the website. ShindouHikaru concluded that although many players had lower Chess.com ratings than USCF ratings, there was no formula he could detect for explaining the relationship.


I got interested in the question at this point and decided to do some additional statistical analyses. I took ShindouHikaru's data, which he had kindly posted on the forum, double checked the USCF ratings and adjusted them for changes in the intervening months. I also added additional data to the sample by using my friends, friends of friends, etc. I only included members if they had active, nonprovisional ratings based on many games. I examined blitz ratings, online (correspondence) ratings, tactics ratings, and USCF regular ratings.


Obviously, this is a nonrepresentative, convenience sample and I would have preferred a large, random sample of chess players from both systems. Nevertheless, the data are still useful and allow us to answer some questions about how the variables relate to each other. In the end, I was able to gather data from 80 people, which I figured would be large enough to detect statistically significant effects. If any Chess.com staff are reading this and curious about these findings, I could do a much more extensive analysis with access to more data...


According to my results, the average Chess.com blitz rating for this sample was 1817 and the average USCF rating was 1945. This supports previous statements that Chess.com ratings tend to be lower than USCF ratings. The difference was 128, which is higher than DrawMaster’s estimate, but lower than AdamRinkleff's. All the variables in the analysis were highly correlated with each other, suggesting that they are all aspects of the same underlying entity of chess skill. USCF ratings correlated r = .83 with tactics ratings, r = .79 with online ratings, and r = .93 with blitz ratings, all p < .001.

Blitz ratings are strongly related to USCF ratings in these data. Honestly, I was shocked at the magnitude of the correlation. Rather than apples and oranges, the situation is more like apples and apples of the same variety, but from a different tree. Then again, the fact that the sample consisted of active, relatively high level players who were serious enough about their Chess.com ratings to put their names on their profiles may have influenced the results. I would expect the correlation to go down with a larger, more casual sample of players.


Because the ratings are so closely related, it makes sense to use regression to calculate an equation to predict one from the other. This produced the following formula: USCF estimate = (Chess.com rating * .93) + 283. Or, if you don’t like math, you could do almost as well with the simpler formula where you just add the difference: USCF estimate = Chess.com rating + 128. Remember these are just estimates based on generalizations. Your mileage may vary.


As a final note, I was perusing the discussion on this issue in the forums and found a post by Pegrin from several years ago. Using Google, he found 59 Chess.com profiles that contained USCF ratings. With the caveat that people’s self-reported ratings might not be accurate, he computed a Pearson correlation of r = .67 and a regression equation of USCF estimate = (Chess.com rating * .74) + 280.5. My data suggest that this would underestimate the USCF rating for serious chess players, but I’ll leave it to the reader to decide which formula works best for them.


So what's my conclusion, for those of you who just skipped to the bottom? According to all the available data, Chess.com ratings and USCF ratings are substantially and meaningfully related to each other. They are clearly the same type of fruit. Also, people generally have lower Chess.com ratings than their published USCF ratings. Finally, the pattern in the data is clear enough that you can get a quick and reasonably accurate estimate of your USCF rating from your Chess.com blitz rating. You can try one of the formulas above, of if you are math phobic, just draw a regression line over the scatterplot with your finger and see where your USCF rating should be.

AdamRinkleff

Yes, probably the reason your number is lower than mine is because of the large number of people you included who have ratings above 2000. The difference seems to shrink at higher levels.

Another factor is how 'active' the player is. I looked specficially at people who are playing chess.com almost every day, and playing in USCF tournaments nearly every week. Obviously, the less frequently someone plays, the more likely the difference will be distorted. In particular, people who haven't recently played in a lot of USCF tournaments will have a lower difference if they have been playing lots of blitz, because they have been improving which isn't reflected within their USCF rating.

I don't understand people who talk about "apples and oranges". I think they simply lack abstract ability. Its easy to see blatant differences between two objects, but its harder to understand relationships, which do exist. People seem to routinely exaggerate the difference between blitz and standard. Although there is one, a relationship exists regardless.

It is also important to note that this relationship holds only for people who play both USCF standard and chess.com blitz. If someone has never played standard, or never played blitz, they will obviously have some initial difficulty adjusting to the different time controls, but that difficulty is reduced with experience.

Its surprising how adamant people are about the idea that chess.com ratings are inflated, rather than deflated. This was the case years ago, but it is certainly no longer true.

Furthermore, since this seems to be a constant source of confusion for people who lack reading comprehension skills: WE ARE NOT TALKING ABOUT 'ONLINE' CHESS. Really, chess.com should change the name of its 'postal' chess, since all the variants here are technically 'online'.

-Adam Rinkleff

AdamRinkleff

Another factor I mentioned in my thread, is that USCF members tend to have lower international FIDE ratings. This is because Americans have below average chess ability. That's not an insult, it's simply a fact. As a result, with the large number of non-Americans on chess.com, we should not be surprised to find that the ratings here are lower than in the USCF.

Ubik42

I would be a pretty extreme outlier on your chart.

bullet -  1089

blitz -    1269

OTB -    1691

DrawMaster

Nice post and replies, and reassuring that several studies - even if limited in scope - seem to indicate something rather similar: chess.com Blitz ratings reasonably reflect playing strength over the board, probably slightly underestimating that latter.

If we go a bit further and look at the published correction equation from FIDE to USCF (USCF = 720 + 0.625*FIDE if FIDE < 2000), and couple this with your correlation, one gets the following:

FIDE = 1.5 * (chess.com Blitz) - 700

This would suggest a sliding scale of parity between FIDE and here, such that:

1000 Blitz = 800 FIDE

1200 Blitz = 1100 FIDE

1400 Blitz = 1400 FIDE

1600 Blitz = 1700 FIDE

1800 Blitz = 2000 FIDE

Very interesting (if my math hasn't failed me). Not sure if there's a spec of validity to it, however. Undecided

GambitExtraordinaire

Very interesting. If the chess.com blitz/uscf ratings are so close, I wonder what would happen if you did a study of the longer time controls?

After all, most players are not playing 5min games at their USCF tournaments.

Ubik42

My impression was the OP was comparing chess.com blitz to USCF standard.

waffllemaster
Ubik42 wrote:

I would be a pretty extreme outlier on your chart.

bullet -  1089

blitz -    1269

OTB -    1691

Is your OTB as established as your chess.com ratings?  e.g. is it really old, new, provisional, etc?

Ubik42
waffllemaster wrote:
Ubik42 wrote:

I would be a pretty extreme outlier on your chart.

bullet -  1089

blitz -    1269

OTB -    1691

Is your OTB as established as your chess.com ratings?  e.g. is it really old, new, provisional, etc?

 

Its established and has been in the 1600's since like foreveor.

I dont do a whole lot of tournaments, but in my most recent tournament (2 weeks ago) my rating went down from 1695 to 1691.

It was as low as 1597 two years ago, when I had been away from chess for like 5 years, but over the course of 3-4 tournaments I brought it back up to the 1690's (so my performance ratio was actually close to 1800 at that time).

I have left chess for as long as 10 years at a time, but every time I come back, my damn rating is in the 1600's.

So it is, sadly, pretty well established.

Suvel

lol another one

Ubik42

Well actual data is always fun. Good OP btw.

waffllemaster
Ubik42 wrote:

Well actual data is always fun. Good OP btw.

+1

(I suppose that's an established rating if ever there was one!)

makikihustle
AdamRinkleff wrote:

Another factor I mentioned in my thread, is that USCF members tend to have lower international FIDE ratings. This is because Americans have below average chess ability. That's not an insult, it's simply a fact.

Totally absurd.

I suggest you look into the actual reason why FIDE ratings are lower for Americans--there are less FIDE rated game opportunities in America than than there are in other countries.

The reason many Americans have higher USCF ratings is because they play in more USCF rated tournaments than FIDE.

Ubik42
makikihustle wrote:
AdamRinkleff wrote:

Another factor I mentioned in my thread, is that USCF members tend to have lower international FIDE ratings. This is because Americans have below average chess ability. That's not an insult, it's simply a fact.

Totally absurd.

I suggest you look into the actual reason why FIDE ratings are lower for Americans--there are less FIDE rated game opportunities in America than than there are in other countries.

The reason many Americans have higher USCF ratings is because they play in more USCF rated tournaments than FIDE.

I like to see numbers, but offhand I cant see a good reason why low numbers of FIDE tournaments mean artificially low ratings. It could just as easily mean artifically high ratings.

One way I suppose, if you wanted to be perverse, is that fewer FIDE competition means chess players here have less chance to hone their skills against top players, but then that kind of supports the OP's point.

waffllemaster

"USCF member tend to have lower international FIDE ratings.  This is because Americans have below average chess ability."

Hmm, I hope you included more than a handful of chess.com users to make such an assessment.

Although it's a silly comment in any case... even among the Americans who do have a FIDE rating you can only include those who are in the same playing pool as the players you're comparing them to ... i.e. no one who lives in the United States.  Why is it that people never seem to remember ratings aren't a measurement like meters or kilograms.  It's a relative measurement against a set of players.

makikihustle
Ubik42 wrote:

I like to see numbers, but offhand I cant see a good reason why low numbers of FIDE tournaments mean artificially low ratings. It could just as easily mean artifically high ratings.

One way I suppose, if you wanted to be perverse, is that fewer FIDE competition means chess players here have less chance to hone their skills against top players, but then that kind of supports the OP's point.


Because you need to play more than just a handful of games to raise your FIDE rating. If a handful of FIDE games a year is all you get, it's easy to see how the FIDE rating can lag behind the USCF rating.

There was an NM who plays here, I can't remember his name. He mentioned that he played in the USCF for 25 years before he finally got a FIDE rating--after moving to Europe.

There simply were no FIDE opportunities in America where he lived.

zltbchess
heres a game for you guys.
1. e4 d5 2. Qh5 g6?? ( 2... Nf6 3. Qd1 dxe4 4. Nc3 Bf5 5. d4 Nc6 6. Bb5 ) ( 2... g6 3. Qxd5 Qxd5 4. exd5 Nf6 5. Nc3 Bf5 6. d3 ) 3. Qe5? ( 3. Qxd5 Qxd5 4. exd5 Nf6 5. Nc3 Bf5 6. Bb5+ c6 7. dxc6 ) ( 3. Qe5 Nf6 4. exd5 Qxd5 5. Nf3 Nc6 6. Bb5 Bf5 7. Nc3 ) 3... f6? ( 3... Nf6 4. exd5 Bf5 5. Bb5+ c6 6. dxc6 Nxc6 7. Bxc6+ bxc6 8. d3 ) ( 3... f6 4. Qxd5 Qxd5 5. exd5 Bf5 6. Nc3 Bxc2 7. Nf3 c6 8. Nd4 ) 4. Qd4? ( 4. Qxd5 Qxd5 5. exd5 Bf5 6. Nf3 c6 7. Nc3 Bxc2 8. Nd4 ) ( 4. Qd4 dxe4 5. Qxe4 Nc6 6. Bb5 Bf5 7. Bxc6+ bxc6 8. Qxc6+ Kf7 ) 4... e5? ( 4... c6 5. Nc3 e5 6. Qd3 d4 7. Nce2 f5 8. Qg3 f4 9. Qb3 ) ( 4... e5 5. Qxd5 Qxd5 6. exd5 Bf5 7. d3 Kf7 8. Nc3 ) 5. Qxd5 Qxd5 6. exd5 Ne7 7. Bc4 c6?! ( 7... Nd7 8. Nc3 Nb6 9. Bb3 Bf5 10. h3 O-O-O 11. g4 ) ( 7... c6 8. dxc6 Nbxc6 9. Na3 Bf5 10. Nf3 e4 11. Nh4 ) 8. Nc3 cxd5 ( 8... Bf5 9. d3 cxd5 10. Nxd5 Nxd5 11. Bxd5 Nc6 12. Be3 Rd8 ) ( 8... cxd5 9. Bxd5 Nbc6 10. Bb3 Bf5 11. d3 O-O-O 12. Nf3 ) 9. Bxd5 Nxd5 10. Nxd5 Be6?! ( 10... Na6 11. Nxf6+ Kf7 12. Nd5 Be6 13. Ne3 e4 14. Ne2 Bd6 15. Nd4 ) ( 10... Be6 11. Nc7+ Kf7 12. Nxa8 Bd6 13. f4 exf4 14. Ne2 Nc6 ) 11. Nxf6+ Kf7 12. Ne4 Nc6 13. b3 Bd5?! ( 13... Nb4 14. Kd1 Be7 15. Nf3 Bg4 16. Ba3 Bxf3+ 17. gxf3 ) ( 13... Bd5 14. Nf3 Nd4 15. Nxd4 Bxe4 16. f3 Bd5 17. Nb5 Bb4 18. Bb2 ) 14. Nd6+?? ( 14. Nf3 Nd4 15. Nxd4 Bxe4 16. f3 Bd5 17. Nb5 Bc6 18. a4 ) ( 14. Nd6+ Bxd6 15. Kf1 Nd4 16. c3 Nc2 17. Rb1 Rac8 ) 14... Bxd6 15. c4?? ( 15. Kf1 Rhc8 16. Bb2 Nd4 17. c3 Ne6 18. c4 Be4 ) ( 15. c4 Bxg2 16. Nh3 Bxh1 17. Ng5+ Kf6 18. d3 Nd4 19. Rb1 ) 15... Bxg2 16. Ne2 Bxh1 17. a4 Nb4? ( 17... Rhd8 18. Bb2 Bc5 19. Ng3 Bf3 20. Ba3 Nb4 21. Bxb4 Bxb4 ) ( 17... Nb4 18. Kf1 Be4 19. Bb2 Rhd8 20. Kg1 Bf3 21. Ng3 Nd3 ) 18. Kf1 Bf3? ( 18... Rhd8 19. Ba3 Bc5 20. Ra2 Bf3 21. Bxb4 Bxb4 22. Nc3 Rac8 ) ( 18... Bf3 19. Ba3 Rad8 20. d4 Bxe2+ 21. Kxe2 Nc2 22. Bxd6 ) 19. Nc3 Bc5? ( 19... Rad8 20. a5 Bc5 21. Nb5 Nd3 22. Ba3 Bxf2 23. Nd6+ Ke6 ) ( 19... Bc5 20. Ba3 Rhd8 21. Bxb4 Bxb4 22. Ke1 Rac8 23. h3 Bc5 ) 20. d3? ( 20. Ba3 Rhd8 21. Bxb4 Bxb4 22. Ke1 Rac8 23. h3 Bc5 ) ( 20. d3 Bd4 21. Bd2 Nxd3 22. Rb1 Bxf2 23. Nd5 a5 24. Nc7 Ra7 ) 20... Bd4 21. Ba3?? ( 21. Bd2 Rhd8 22. Rb1 Nxd3 23. Nd5 Bxd5 24. cxd5 Bxf2 25. Ke2 ) ( 21. Ba3 Bxc3 22. Rc1 Bd2 23. Ra1 Rhd8 24. Bxb4 Bxb4 25. Rc1 Rxd3 ) 21... Bxc3 22. Ra2?? ( 22. Rc1 Bd2 23. h4 Rad8 24. Ra1 Nxd3 25. Ra2 Bf4 ) ( 22. Ra2 Nxa2 23. Kg1 Rhd8 24. h4 Rxd3 25. Kh2 Nb4 ) 22... Nxa2 23. c5 Bb4? ( 23... Rhd8 24. Kg1 Rxd3 25. h3 Bd4 26. c6 Bxc6 27. Bd6 Rxb3 28. h4 Rb2 ) ( 23... Bb4 24. Bxb4 Nxb4 25. Ke1 Nxd3+ 26. Kd2 Nxc5 27. Ke3 Bd5 ) 24. Bb2? ( 24. Bxb4 Nxb4 25. h4 Nxd3 26. Kg1 Nxc5 27. b4 Nxa4 28. Kf1 Rhd8 ) ( 24. Bb2 Rhd8 25. Kg1 Rxd3 26. h4 Re8 27. Kh2 Rxb3 28. Ba1 Bxc5 29. Kg3 ) 24... Bxc5? ( 24... Rhd8 25. h4 Rxd3 26. Kg1 Re8 27. Kh2 Rxb3 28. Ba1 Bxc5 29. Kg3 ) ( 24... Bxc5 25. Bxe5 Rhd8 26. d4 Bxd4 27. Bf4 Bd5 28. Be3 ) 25. Bxe5?? ( 25. h3 Rhd8 26. d4 exd4 27. a5 Re8 28. a6 bxa6 ) ( 25. Bxe5 Rhd8 26. Bd4 Rxd4 27. b4 Nxb4 28. a5 Nxd3 ) 25... Rhe8? ( 25... Rhd8 26. Kg1 Rxd3 27. h4 Re8 28. Kh2 Rxe5 29. b4 Nxb4 30. Kg3 Bc6+ 31. Kh2 Bxf2 32. a5 ) ( 25... Rhe8 26. d4 Bb4 27. h3 Rec8 28. Kg1 Nc1 29. h4 ) 26. Bb2?? ( 26. d4 Bb4 27. h3 Rac8 28. Kg1 Nc1 29. Kh2 Bd5 30. Kg3 ) ( 26. Bb2 Bb4 27. Be5 Rxe5 28. d4 Re1# ) 26... Be2+? ( 26... Bb4 27. Be5 Rxe5 28. d4 Re1# ) ( 26... Be2+ 27. Kg2 Bxd3 28. Kg3 Re2 29. Ba1 Rd8 30. f4 ) ( 26... Be2+ 27. Kg2 Bxd3 28. Kg3 Re2 29. Ba1 Rd8 30. f4 )
AdamRinkleff
waffllemaster wrote:

"USCF members tend to have lower international FIDE ratings.  This is because Americans have below average chess ability."

Hmm, I hope you included more than a handful of chess.com users to make such an assessment.

Its blatantly obvious. You can deny it if you want, but it's true. Germany, all by itself, has more grandmasters than the United States.

SmyslovFan

JcButler, 

Thank you for a thoughtful, well-reasoned statistical analysis. It makes sense that there is a correlation between rating groups since they use similar methods to calculate ratings. 

I am surprised though at how closely the ratings here match USCF ratings. Blitz chess at home is subject to so many interruptions. I figure that about 1 game in 10 is decided by mouse slips or connection problems. That alone would make comparisons more difficult.

And apart from the time control, the playing conditions are very different. How many people have lost blitz games due to interruptions that they'd never have to worry about in a tournament? And of course, chess.com blitz is far more casual in other ways. 

I wonder how someone who took chess.com's blitz chess as seriously as he did a USCF/FIDE tournament would perform here. 

Again, thank your for taking the time to make this study!

GambitExtraordinaire
GambitExtraordinaire wrote:

Very interesting. If the chess.com blitz/uscf ratings are so close, I wonder what would happen if you did a study of the longer time controls?

After all, most players are not playing 5min games at their USCF tournaments.

Ubik42 wrote:

My impression was the OP was comparing chess.com blitz to USCF standard.

Yes, that was my impression as well. I might have been more clear, I suppose.

I meant, a more interesting study might be the comparison of chess.com standard/USCF ratings, instead of chess.com blitz/USCF ratings, since most USCF rated games are longer time controls.