Unfair Rating Inflation Due to New Puzzle Rating System

Sort:
Avatar of Martin_Stahl
AvatarX13337 wrote:

On this brand-new account I went from 800 to 2700 in a few days. My highest puzzle rating on the other account was 2684, and that was quite a challenge. I could grind this rating higher forever if they only give me 1800 puzzles.

On the old system I could go from 400 - 2500 plus in a few hours of puzzles.

Avatar of GraysonKellogg
Martin_Stahl wrote:
AvatarX13337 wrote:

On this brand-new account I went from 800 to 2700 in a few days. My highest puzzle rating on the other account was 2684, and that was quite a challenge. I could grind this rating higher forever if they only give me 1800 puzzles.

On the old system I could go from 400 - 2500 plus in a few hours of puzzles.

You’re also pretty good at the game (1900, congrats!) so that probably has something to do with it. This thread is more about people who are rated below 1900 effortlessly getting to the kinds of ratings that professionals had to work for.

Avatar of Martin_Stahl

I'm 1900 in daily, which is an older rating and probably around 100 higher than it really should be. I peaked near 1650 OTB but there's no way I'm actually 2500+ strength in tactical positions really.

My puzzles have fluctuated between 2500 and 2900 a lot, and even with the updates, I'm seeing very similar ups and downs. My understanding of some of the back-end changes, which difficulty isn't really part of, is that they're aimed at having ratings be more realistic and more accurate.

There have been numerous changes over the years with the way puzzles work. That includes changes in the rating system, puzzle selection algorithm, etc. This change feels more drastic than the others, especially in conjunction with the changes in the backend ratings, but so far I know I'm still feeling challenged, which is one of my primary wants in the puzzle system.

Switched to Extra Hard after a recent drop and a string of +5's but I'm still getting challenging puzzles.

Avatar of MiralSelchintet

Not sure why chess.com every changed this. It's like Bonds and steroids. I worked to reach 2600.

One's puzzle rating means very little now. Playing two dozen 1100 rated puzzles will inflate a 2500 puzzle rating.

Avatar of pwd79

Behaviour has changed again for me now. Consistently serving puzzles between about 1400 and 1600 on the default setting. It had been varying more widely with some sub 1000 puzzles and some over 2000.

Think my rating peaked around 2700 before the changes. I'm over 3600 now. Best puzzle I've ever solved is a few hundred lower than that.

Like many others, I can still solve the puzzles I'm served consistently enough to continue to inflate my rating indefinitely under the current system.

I wonder if it is a business decision to deliberately link ratings with a paid feature (volume of puzzles done).

Avatar of HarmlessAlien

I got up to 3450 on my own without the update, seems to me that I would be able to get to 4000 at this point with almost 0 effort which is a little unnerving. I gained 50 points today doing puzzles rated 1000 points below my rating. Can't get anything higher rated than 2450 (on extra hard). Regular gives me 2150ish rated puzzles(!?) am I going to have to get to 4500 to be able to start getting 3k rated puzzles again?

Anyway under the current system ratings mean almost nothing, a little but frustrating that all the hard work means nothing now I don't know if there was some sort of problem with the old system but this one is definitely not "better" unless chess.com is trying to get us to pay for puzzle rating (volume of puzzles)

Avatar of TBM

I agree with this post. I can reach 10,000+ fairly easily with the new update. "Extra Hard" is not hard at all. Chess.com can defend this terrible update, but it's clear the rest of the community disagrees.

Avatar of FarArden

Where are chess.com defending the change? I've looked for their commentary but haven't found anything?

Avatar of pwd79

They aren't defending it. They are waiting to see if it is accepted.

It is a business decision. Your puzzle rating is now directly linked to how many puzzles you do. How many puzzles you can do is linked to paying for membership. So it is an incentive to pay for membership.

They aren't defending it because they don't want to come out and say this.

Avatar of FarArden

Well, if they are reading. I have a diamond membership and have stopped doing puzzles on the site. Which I used to love.

Avatar of eddiesc03

Likewise I paid for a year but ain't paying again if puzzles remain f*****

Lichess here we come...

Avatar of Martin_Stahl
pwd79 wrote:

They aren't defending it. They are waiting to see if it is accepted.

It is a business decision. Your puzzle rating is now directly linked to how many puzzles you do. How many puzzles you can do is linked to paying for membership. So it is an incentive to pay for membership.

They aren't defending it because they don't want to come out and say this.

There are additional changes coming and when those happen there should be some communication around the overall changes.

Avatar of Maria
Martin_Stahl wrote:
pwd79 wrote:

I think there is a static +4 or +5 on ratings adjustments, resulting in ranking inflation for years.

Now it has become obtrusive for anyone over 2200 or so, as most of the puzzles it serves by default are under 1900, but you still get a guaranteed 5 for solving them, regardless of rating difference or time taken.

Seems like it could be easily fixed by removing the inflation and requiring a puzzles to be no more than 500 points lower than your rating to affect it.

How to retroactively fix all the ballooning ratings is another question. I think I'd like a dual ratings system. You can see your ballooning ranking with the current approach with one score, while another has your score with the inflation removed (starting from your rating before the change if applying that retroactively isn't possible).

I quite like the ballooning rating for what it is, even if it is absurd if taken as a genuine rating, and it undermines whatever ratings people have achieved before the change.

In my last 25 puzzles, on Hard, I have had

7 - 2300+

6 - 2100-2299

5 - 1900-2099

2 - 1500-1899

5 - < 1499

The listed puzzle ratings in the UI are old and not used in the selection or rating algorithm, as I understand it, so I can't say for sure how off they are really. Those should be updated in the near future

That said, the distribution, on the shown rating looks decent and since the change I've been getting around 24-29% wrong. On the old system I was close to 50/50. Extra Hard would likely put me closer to that.

I'm still iny historical rating range, 2500-2900. My last session was a good streak so It's possible I could climb towards my highest before the update, at over 3100, but I'm not sure

In my opinion, so far, I'm still getting challenging puzzles, and while my solve percentages are higher, point increases aren't dramatic and within normal range.

Understandably, that's not true for everyone and for Standard difficulty, it's not going to be great for some. Extra Hard should be more in line with the older, pre-difficulty setting.

I am afraid not. Changing the difficultly Hard to Extra Hard just gave me 1900 puzzles.

Avatar of Martin_Stahl
paristarr wrote:

I am afraid not. Changing the difficultly Hard to Extra Hard just gave me 1900 puzzles.

There' was a different change that happened about a week after that quoted post. More recently the range has mostly been smaller in most cases

Avatar of umbravolt

Got from 2600 to 4200 in a week, so yes the rating system has been extremely inflated. I used to get puzzles rated 2300 - 3000 but now even with extra hard mode on I get max 2100 rated puzzles.

Avatar of Richard

Hi all,

Thanks for your patience and understanding. We recognize the concerns that have been raised and truly apologize for the friction this situation has caused.

Our team has issued an official statement to address the matter directly. You can read it here:
🔗 https://x.com/chesscomsupport/status/1949970540845793609

If you have any further questions or feedback, please feel free to share, we’re listening. 💪🏻

Avatar of wuestenigel
Richard hat geschrieben:

Hi all,

Thanks for your patience and understanding. We recognize the concerns that have been raised and truly apologize for the friction this situation has caused.

Our team has issued an official statement to address the matter directly. You can read it here:
🔗 https://x.com/chesscomsupport/status/1949970540845793609

If you have any further questions or feedback, please feel free to share, we’re listening. 💪🏻

Thanks for the update! As a puzzle enthusiast, I’m excited to see what the coming weeks have in store for us! 🧩
PS: Any idea why I don't find my puzzle rating on my profile page @Wuestenigel?

Avatar of Martin_Stahl

There's a limited number of spaces there; you can click the Stats tab to get to the Puzzle rating if it's not otherwise showing.

Avatar of Maria
Martin_Stahl wrote:

I'm 1900 in daily, which is an older rating and probably around 100 higher than it really should be. I peaked near 1650 OTB but there's no way I'm actually 2500+ strength in tactical positions really.

My puzzles have fluctuated between 2500 and 2900 a lot, and even with the updates, I'm seeing very similar ups and downs. My understanding of some of the back-end changes, which difficulty isn't really part of, is that they're aimed at having ratings be more realistic and more accurate.

There have been numerous changes over the years with the way puzzles work. That includes changes in the rating system, puzzle selection algorithm, etc. This change feels more drastic than the others, especially in conjunction with the changes in the backend ratings, but so far I know I'm still feeling challenged, which is one of my primary wants in the puzzle system.

Switched to Extra Hard after a recent drop and a string of +5's but I'm still getting challenging puzzles.

huh, that's quite strange. all my puzzles were 1900, now they dropped to 1800 even though my difficultly is always on extra hard (yes, I am sure I saved it to that.)

Avatar of wuestenigel
Martin_Stahl hat geschrieben:

There's a limited number of spaces there; you can click the Stats tab to get to the Puzzle rating if it's not otherwise showing.

yes, but I want it on my main profile as many other players have. There is no need for an empty "Daily" section.