Ratings are meant to measure relative strength within a pool of players.  In USCF tournament play, for example, if you sit down with a 1900 player, you can fairly reliably expect a certain level of game.  Informally surveying the 1900 players on chess.com, however, reveals a wide range of playing at that level.

 

The reason for this I have seen most often is that the hypothetical 1900 player of whom I speak is not required to try his skill against other 1900 players.  A 1900 player may have an average opponent rating of 1000, or an average opponent rating of 1900.  These two players will play very different games.  I do not think the difference in average opponent strengths is taken into account by Glicko.

 

I'm wondering if there is some way to give chess.com ratings a reality check, in other words, to help ensure internal consistency among chess.com ratings.  For example, could ratings be impacted by playing or not playing opponents within a certain range of one's own rating?  If a player declined to play any opponent within that range during a certain number of his games (since we are all engaging opponents at vastly different rates), his rating could go down by a value determined by the chess.com staff in consultation with Prof. Glickman.

 

Any thoughts out there about this?  Erik and other honored chess.com staff please help out here! 

I agree with some of what you're saying. Most people here seem to have played relatively few games, so I imagine that the ratings will stabilise as more people join and more games are played. I don't think that Glicko is a factor in the unreliability though -- FICS uses Glicko and their ratings seem to be quite stable and reliable.

I have noticed that the ratings here seem to be higher than on other sites. I've played a lot on FICS and ICC and my rating is about 100 to 200 higher here than on those sites. Has anyone else found the same?

As far as the influence of pairings  on the ratings go, the only truely fair way to do it is to do random pairings where you get no choice over who you play. ICC does this with its '5-minute' rating category where you play a random opponent, as opposed to the 'blitz' category where you can play whoever you like.

Random pairings mean that you can't play your friends and can't do rematches though, so there seems to be no perfect solution.

To me it seems ok as it stands though -- on sites like these your rating is just a guide, it doesn't mean all that much.

I don't think it really matters.  And Chelex is right.  I've always maintained that the CHESS.COm rating only makes sense here.  It's a pretty pointless exercise trying to align it with OTB ratings.  This point has been done to death on various other postings, so lets just park that one there. 

 

You do have a point though, and I think some ratings maybe artificially high due through the circumstances you mention.  However, the hypothetical 1900 player you mention will be found out very quickly when playing someone with a solid 1900 rating.

 

Don't worry about it

I have to agree... I have a game in progress right now against a 1900 ish player who has never beat anyone over 1700.. yes it's free rating points for me because I'm one of his rare 1800 games (assuming I win/draw) but the rating is clearly inflated.

 

This is somewhat tangential to the original point, but I think there needs to be a tweaking to the "average opponent" statistic or maybe an additional statistic. A game I played 50 or 100 games ago doesn't really affect my rating today, but it still affects my average opponent rating. It would be nice to have an average opponent statistic that had a recency weighting similar to the rating does instead of every opponent being weighted equal.

 

For example, my average opponent on chess.com has been 1735. That's averaged over every game I have played. But if you look in the last month, only one rated game I've played has been against a player rated lower than that, and most of them with players 100+ points higher. So someone who looks at my rating and thinks I earned it against an average opponent of 1735 is being misled. 

ih8sens wrote:

I have to agree... I have a game in progress right now against a 1900 ish player who has never beat anyone over 1700.. yes it's free rating points for me because I'm one of his rare 1800 games (assuming I win/draw) but the rating is clearly inflated.

---

 Your conclusions don't really follow. If he is rated 1900 and you are rated 1800, then even though he has rarely played anyone over 1700 he is still likely a better player. Why? Because clearly he beats 1700s more consistently than you do. If your record aganist 1700s was as good as his, your rating would be as high as his. Since it's not, it is statistically more likely that he is a better player. This is true even though some of your rating is earned against higher rated players.

Chelex wrote:

I have noticed that the ratings here seem to be higher than on other sites. I've played a lot on FICS and ICC and my rating is about 100 to 200 higher here than on those sites. Has anyone else found the same?

---

 Really? Actually my rating on FICS is about 200 points higher than my Chess.com rating. Then again, I don't really think in my Chess.com games cuz I don't care that much =P 

my rating is 1776 right now.  i played a chessmaster game the other day in this range and got my butt kicked

The way ratings are calculated Loomis should be right. Based on the differences between two players ratings it should be possible to roughly determine the odds of either player winning. Based on their outcome the game their ratings are then adjusted accordingly. When you play someone with a lower rating your odds of winning are better, but you're putting more rating points on the line. Supposedly this all works out. I wonder, though, if there's any empirical test to show that it does.

My rating has been between 1800-1900 for a few months. And Nearly every opponent I play is around 1700-1900. My average opponent seems to be close to 1580, because I once played a game against a 900 rated, and 1300 rated player when I just joined. Then I think evaluating my playing strength by average opponent can by quite misleading as well.

Perhaps, if players wanted more accurate ratings, the system would only allow a rated game within a parameter of your personal rating.  For example; if your rating is 1500, then the only rated games you play would be within 100 points, give or take (1400-1600).  Any games played outside of that parameter would be unrated games.  Of course this would all be an automatic feature, unless you specifically wanted to play an unrated game.  This would allow players to pick and choose challenges more accurately.  This could also allow the 'Avg. Opp.' stat to be dropped.

        i think over time things will get more accurate. a good way to make things better would be to have a few chess engines that play at different levels and have them be able to be challenged by other players. so say there is a chess engine that has a known strength of 1800, you put it at that rating and lock it at that rating. the people who challenge the chess engine will have there rating changed but not the engine because we already know it plays at 1800.

excalibur8, I agree...

 

If you want true ratings, I suggest a web site or organization that will give true results. (And, I dare say, most of the players on this site will see their ratings substantially plummet!)

 

If you're here playing at Chess.com, play for the enjoyment of it. I personally am here to play a friend or two in a more convenient fashion over the web, as well as to play with other players around the world. I do look at another players rating and stats, but more out of curiosity than for strategizing my own rating or reputation.

 

Have fun.

 

Peace.

    Ratings, though the concept started to keep players of equal talent grouped together, has become a tool to stroke the ego. A player who wants to pad the stats so he can say he is rated higher than another will. One who earns his rating does it so he can say he earns it. On this site, I was given the 1200 rating, no one knows who I am and no one knows my true rating. I visit these sites to gain scraps of knowledge that I may not gleam from pouring over the same books over and over and to get the view points of people who are truly students of the game.(again ego). If you love the game the petty squabbles are just a nuisance. If your ego demands that you be recognized for your endeavours. Then create something that will make all the squabblers satisfied...

Why are the ratings here higher than in other places?  On gameknot they seem lower or more in keeping with Fide.  Is it mainly people haven't played that many games yet?  If you play someone so much below you though, you really can't ever lose or it would make no sense, you would lose so many points at 1900 playing a 1000 person.  Anyone doing this is a little weird anyway, someone who never ever wants to lose playing much infereior opponents.

Loomis wrote

---

 Your conclusions don't really follow. If he is rated 1900 and you are rated 1800, then even though he has rarely played anyone over 1700 he is still likely a better player. Why? Because clearly he beats 1700s more consistently than you do. If your record aganist 1700s was as good as his, your rating would be as high as his. Since it's not, it is statistically more likely that he is a better player. This is true even though some of your rating is earned against higher rated players.

---

 My average opponent is stronger than his... I'm also relatively new to the site so my rating isn't quite accurate.. and he doesn't beat 1700's more consistently than me.. in fact, I believe he's only beat 1...  That's my point, his rating is overinflated because he only plays weaker players than himself...

 edit - if a 1700 player only plays 1400 he'll win consistently and gradually increase his rating while I play 1800's and 1900's, sometimes losing, sometimes winning.. my rating stays about static at 1800-1850 while his gradually increases 1 point at a time.. even though were we to play each other he could be 2000 while I'm still 1850 and I would presumably beat him.

this is an important topic and i appreciate the comments.

first off, to understand ratings you have to understand the math behind them. for those of you who have not read my article on the topic, go here. 

please consider that a rating is ONLY valued within the context of the rated group. so chess.com ratings are chess.com ratings - not FIDE, not USCF, not other sites. but the way ratings are done on chess.com are mathematically correct. the fact that ratings on chess.com are not the same as USCF/FIDE/other sites is that we don't have the same spectrum of players :) we have, so far, attracted more NEW chess players than other sites and fewer GMs, etc. also, our site has only had online chess play for 7 months - not decades.

there are a FEW things we could do to adjust the ratings on chess.com. we could start people at LOWER ratings (say 1000).  

i also agree that the average opponent ratings, best win/loss, and a few others should be trailing (say last 90 days).  i'm going to start a new thread on stats recommendations.

please ask any questions and make any suggestions on ratings here: i would love to hear them! 

Part of me heartily agrees with the observation implicit in the question, What's the big deal?--after all, it's just a game.  But if the ratings have no meaning, why have them?  Erik's concern for the validity of the stats is an important part of the robustness of chess.com.  I'm looking forward to posting to his new thread.

maybe you could use the avarage rating of opponent, compare it to your rating and if your rating = (opponent rating + x), x being 100,200 or a certain % this player will automaticly get challanges from: (for example 3 players (with that same "problem") of his own rating ( so he can't cry that he already played that guy or blabla). refusing all 3 challanges will lead in a lose of rating or a +0 untill every win untill you play someone of your own rating range( so you can still choose who you play against)

if you really want to clearly say that chess.com ratings are not comparable to others then do something like moving the decimal over one so someone rated 1785 would be 178.5. it would be the same but at the same time you would think it is not comparable. In Arizona when i played for my high school the state would use the same rating system but they would changed the end result basically by dividing by two and subtracting 200. so 1800 then divided by two 900 then subtracting 200, 700.