Misrated Puzzles

Sort:
Avatar of DubiousChess

I have a puzzle rating of around 2600. I've noticed recently about half of the puzzles I get are unbelievably hard, to the point of them being essentially impossible for a 2600 to solve. They have pass rates of anywhere of 0.0% to 1.0%. If a puzzle given to 2600 rated players is only solved by anywhere from 0 to 10 in 1,000... then that puzzle should be rated more like 3500 or 4000.

I thought the way puzzles worked was that the intended pass rate is around 50%. If it's higher than 50%, the rating of the puzzle is lowered and it's given to lower rated players. Conversely, if a puzzle has an insanely low pass rate, the puzzle rating would then go up until it's around 50%.

The way it is right now, I've essentially hit a wall at 2600 since even if I have a 100% pass rate against the passable puzzles, I lose tons of rating from the unpassable ones.

Avatar of Martin_Stahl
FastAndDubious wrote:

I have a puzzle rating of around 2600. I've noticed recently about half of the puzzles I get are unbelievably hard, to the point of them being essentially impossible for a 2600 to solve. They have pass rates of anywhere of 0.0% to 1.0%. If a puzzle given to 2600 rated players is only solved by anywhere from 0 to 10 in 1,000... then that puzzle should be rated more like 3500 or 4000.

I thought the way puzzles worked was that the intended pass rate is around 50%. If it's higher than 50%, the rating of the puzzle is lowered and it's given to lower rated players. Conversely, if a puzzle has an insanely low pass rate, the puzzle rating would then go up until it's around 50%.

The way it is right now, I've essentially hit a wall at 2600 since even if I have a 100% pass rate against the passable puzzles, I lose tons of rating from the unpassable ones.

 

It's very likely the puzzles are newer and haven't settled to a good rating yet. Ratings of puzzles change by people solving them.

Avatar of DubiousChess

The new puzzles, which usually have a similarly low pass rate, are listed as unrated, and give you +1 rating regardless of whether you get it right. I'm not talking about those ones - I'm talking about the ones that have an established rating, and penalize you fully for getting it incorrect. So if these are "newer" puzzles, then they are being changed from unrated to rated far too soon. They should stay unrated until they have a reasonable pass rate, and therefore a reasonable rating.

 

Also, like I said, recently this has been close to 50% of the puzzles I get. It's hard to believe 50% of puzzles are brand new.

Avatar of Martin_Stahl
FastAndDubious wrote:

The new puzzles, which usually have a similarly low pass rate, are listed as unrated, and give you +1 rating regardless of whether you get it right. I'm not talking about those ones - I'm talking about the ones that have an established rating, and penalize you fully for getting it incorrect. So if these are "newer" puzzles, then they are being changed from unrated to rated far too soon. They should stay unrated until they have a reasonable pass rate, and therefore a reasonable rating.

 

Also, like I said, recently this has been close to 50% of the puzzles I get. It's hard to believe 50% of puzzles are brand new.

 

The site changed how quickly pending puzzles move to regular puzzles recently, but even after puzzles drop out of pending state, their ratings still change based on being solved.  3 out of 4 of your last puzzles where you got 0% that have low pass rates, all of them have very low numbers of solves. So those certainly were ones that had recently been pending.

Avatar of DubiousChess

So the issue then is that they're just being moved from the pending state too early.

Avatar of Torsen

I have also experienced this quite a lot.

Avatar of shirazu0

The rating system is just completely broken right now. I actually think there is something wrong with the code somewhere. The only puzzles with accurate ratings are those that are very old.

Avatar of Martin_Stahl
shirazu0 wrote:

The rating system is just completely broken right now. I actually think there is something wrong with the code somewhere. The only puzzles with accurate ratings are those that are very old.

 

The broken part appears to be that low pass rate puzzles are likely being released out of pending state too early.  Once released, the rating will continue to fluxuate based on being solved/failed.