Tactics Trainer Puzzle Rating

The autobalancing function of the puzzle rating for multimove puzzles is broken.

You might explain why you think that and whether you are doing the tactics on the app or v3 site or, on the v2 website.

I'm on the v2 website.

The puzzle rating is supposed to autobance depending on how difficult the puzzle is. Here's an example where the autobalancing doesn't work as intended:

http://www.chess.com/tactics/?id=83948

I guess you are saying that due to its low rating and low pass rate? The auto-balancing takes into account the rating of the people playing against it and the time to solve. 

https://support.chess.com/customer/en/portal/articles/1444811-how-does-the-time-and-ratings-work-

With such a low pass rate it must be that people are getting most of the puzzle right but are missing something significant. So the rating isn't changing much, since people are getting partial credit but are still failing overall.

I actually haven't attempted it yet, so that is speculation. It probably should be modified some,  since a low pass rate indicates something is a problem

That's exactly what I mean! 

Here's a list of other puzzles that are rated too low, there are quite a few thousands:

http://www.chess.com/tactics/problems?sortby=pass_percent_lowest

I finally did that one puzzle and I can see why it has a low pass rate and still why the rating is low. The last move is a harder one to find. 

It is possible that no credit should be given without all the moves in that case. That would likely boost the rating and eventually the pass rate. 

Thank you for understanding.

In Wikipedia, Tactics are explained to be a sequence of moves. In that case, do you think partial credit should be given in multi-move puzzles at all?

Well, I think it is a balance overall. In longer puzzles, may people would get upset getting most of it right to fail on the last move. But, there should probably be a cutoff or you get similar results.

Of course, some of the puzzles with low pass rates, may just not be very good puzzles overall. Not sure if staff looks at those or not.

Mathematically, it's better if the puzzle was counted as a whole. But if there was a cutoff like you suggest, it would also be a lot better.

There was an update to the formula early February, and after that it seems multi-move puzzles are scored much better. I'm getting lots of minus-points every time I fail to solve all the moves. Great.

I only have data from my own puzzles, and there are limited times that I fail a multi-move puzzle within 2x average time. So it will take a long time for me to gather enough data to crack the formula. If anyone knows the complete formula, pls post it.

One thing I noticed straight away with the new formula, is that it always rewards me with 1 ratingpoint whenever I solve a puzzle and the time has gone out. What am I getting the rating point for?

I have climbed 150 rating points in the last weeks because of this. Puzzles are so hard now, they take me approx 5 minutes each, when they used to take a minute.

The self-balancing part that was broken with multi-move puzzles seems to have been fixed. Now the self-balancing is broken with every other puzzle instead.

And how come it seems I am the only one who seem to care about the formula, who also at the same time understand the formula. I hope you understand, I'm not typing all this to complain. I'm trying to help you guys make the site better.

The point is for solving the puzzle correctly. If it takes too long, maybe it would be better just give no points. I imagine it will be balanced more as the more people do tactics and they see how the ratings are impacted.

As to the formula, I would guess they just tweaked the one I posted earlier. Since v3 isn't fully live and only the app and v3 of the site use the new formulas, they may not update the article above until the site goes live for everyone as the default.

I did some tactics via the app earlier today and I seemed to have a good balance, even if I did fail too many

I have gathered more data, and unfortunately it shows that my hopes were too high about the formula change. I still get rating points for playing multimove puzzles too quickly and failing them.

For example: #522556, Solved 2/3 moves in 51% of average time: My rating goes up by 2 points.

Essentially TT is telling me "You failed to see the tactics in this puzzle. You must be good! Here's 2 rating points!"

It's like losing a rated game against another player, and then after the game, chess.com goes:
"You just lost to checkmate after a devastating blunder. But you played very fast, and very well in the opening! Here, have some rating ponts from the other player!"

And nowdays also TT keeps telling me "You spent an extraordinary amount of time solving this puzzle, far, far longer than the average solving time on this puzzle. You must be good! Here's a rating point!"

It's like losing a rated game against another player, and then after the game, chess.com goes:
"You just lost on time. But you played some very good moves and you have a winning position. Here, have a rating point from the other player!"

Does this seem logical to anyone?

Imagine you are the winning player in the above scenarios. In what way is it logical now?

Some tactics give points for getting the majority of moves correct. I don't know what the cut-off is but not every tactic is like that.

There is no cut-off. If you solve 3 of 5 moves, the formula gives you a success of 3/5 (aka 60%) of what you would get if you solved all 5.

1. Rating change for the puzzle and for the player are calculated different. That means puzzles will no longer be in sync with the average rating of those who solve them.

2. Solving a puzzle always gives you a minimum of 1 rating point, no matter how long time you spent. The problem with this is: The puzzle is (supposed to be) rated at the average level of those who solve them. And you are rated (approx) the same as the puzzle. So in extension, you are actually solving the puzzle in competition with other players who are rated the same as you. Now if your solving time is longer than the average time other players use, you are a worse player than them. So TT should reduce your rating, not increase it.

Scared of angry folks getting negative rating points after solving a puzzle? Ok, let's meet half way. Give them 0 points, not 1.

3. Multimove puzzles scoring is calculated wrong. Failed puzzles are credited if you get the first moves right.  A tactic is a whole sequence of moves, you can't give vredit for getting some of them right. If you only get some of them right, then you didn't understand the tactic. If you don't understand the tactic, it's too advanced for you. Remove some rating points, and play an easier tactic instead.

4. It's not "My Score". It's "My RATING".

Just adding to this thread. Nobody seems to be reading this, so I don't know why I bother. My dream is that one day, someone will read this and say "Hey! he's right. We've been blind. Let's change our broken rating system so it works."

Well if you do, please contact me. I'm willing to assist.

Just a copypaste of one of my replies under one of the tactics.

You are right that the scoring system changed. Unfortunately not for the better.

There were a few problems before. They fixed one or two, but there are a couple new ones instead.

The whole point of the rating system is to balance puzzles and players, so that you can follow your progress, and so you don't get too hard or too easy challenges.

Average time is an important variable in the scoring system because A: Chess is a timed game, the faster you think, the better player you are, B: to help balance the puzzles, and C: make sure puzzles don't take too long to solve

The Average time actually is a representation of how long time players can be bothered to think about a tactic. If you were allowed more time, you would solve harder puzzles. This would cause the your rating to go up and the puzzle rating to go down. That means you will face puzzles that are harder and will take longer to finish. Personally I liked TT when a puzzle took 1 minute on average, and I could usually understand the solution. I don't like it when it takes 5 mintes and I still don't understand it.

Players like me, who like to solve the entire puzzle in my head before making the first move, will likely have rised a lot in rating after the formula change. Mine went up almost 200 points. It's good for my self image, but solving a 1450 puzzle in my head takes far longer than solving a 1250 puzzle. Now I find myself on a daily basis facing puzzles where I frustratingly stare at it for 10 minutes making no progress in my head whatsoever about which piece to move where.

Negative points, even when solving a puzzle, is counterintuitive. But it's important as a rating and blancing instrument. Consider that the puzzle rating is a representation, or extention if you like, of the rating of the players that played the puzzle before you. Let's call these average players Mr Average. If you spend a long time, far longer than Mr Average, then that means you solve it poorer than him. It means he beat you at the puzzle. In chess, his rating would go up and your down.

Not so with the new formula. With the new formula, you get 1 rating point, even if Mr Average beats you. It unbalances the whole rating system.

So I for one, do not cheer the new rating system.

Oh, and if you don't like puzzles with low pass rates, there's a perfectly good reason why. TT staff f***ed up when they decided to give partial scoring for multimove puzzles.

There's a simple fix, it would actually simplify the scoring formula.

I have unfortunately given up on explaining to them. It seems logic, reasoning and maths are not the right tools wether I try support or the forum. 

I manage to gain their attention, but then the threads just die out. Maybe I bore them to death with my explanations.

You've really sweated over the details in all of this.  Good for you!  Unfortunately, as you suspect, basically no one gives a sh#t.  It's just too involved and complicated for the average member (and maybe even the above average).  Sorry.

I'll read the whole post in a little bit but they changed the scoring based on a lot of feedback.  If enough people dislike the new methods they may look at tweaking them again.

All that really matters is if you get tactics appropriate for your level. If you are still getting those then there really isn't much of problem

I am also interested in this. Saw your long post that you quote above here:

https://www.chess.com/tactics/493522?ncc=1#first_new_comment

I don't have time just now so I am posting this just so that I can track this thread. Thanks.

Baddongo, I agree, it's too involved for the average member. But there must be someone there sharing my interest in game theory, or maths. I thought players of chess, the most logical game of all, would be inclined to those interests, but it seems I'm seriously mistaken.

I thought also it would be interesting for staff to find a way to fix a broken formula that causes frustration among the players.