Tactics Trainer - change in scoring?

Sort:
JillKovacs

Since yesterday, the tactics trainer doesn't increase your score for a win nearly as much as it did previously.  I'm talking about how when you finish and win in in a very small fraction of the average time, but still only score 60 something percent, and only get a few points.

I'm not hung up on a score per se, but for comparative purposes, and for that graph of my tactics training score to be meaningful, there needs to be consistency.

Has chess.com changed the way scores are calculated? Just want to know. Thanks.

Jpatrick

You are right.  There have been some adjustments to the rating/scoring in Tactics Trainer.  For example I notice that the score for solving a problem after time expired is now 10% instead of 20%.

There's still a lot of work that needs to be done in this area.  For one thing, the rating distribution of the problems is multimodal.

Not necessarily independent of that multimodality is that the Rd value of the problems are unrealistically high, even after quite a number of rated attempts. These Rd values should converge to a low number (say less than 10) after a reasonable number of tries (say 500).  It makes no sense to have time dependence of Rd for the problems since their "skill" doesn't change.

I could go on, but you get the idea.

Bottom line is that it might be necessary to reset everything: problem ratings, player ratings, and all once appropraiate algorithms are in place.

JillKovacs

Thanks. I sort of get the idea, though I'm not sure what RD means.  I really get intimidated by all of the lingo related to chess. I used to just play chess with my dad when I was a kid and then started playing here about a year ago. :)

Anyway, thanks for your feedback.

prouddad

I also just noticed the change in the scoring algorithm.  Seems like chess.com staff dropped the ball in informing users of the change and the reasons for it.  I also agree with jpatrick that the acoring algorithm is less than perfect.  One of my pet peaves is a problem with a 5% pass rate is rated 1300 because most people get 5 out of 6 moves.  In these types of problems the 1st 5 moves are "easy" and make it qualify as a 1300 problem but its the 6th move thats hard to find and make it qualify as a 2000 problem.  A suggestion as to how to handle this is to discontinue the practice of "partial credit".  That would allow the current self adjustung rating system to properly bump problems like this up to their correct rating. 

RetGuvvie98

the problems' ratings "float" just like player's ratings "float" for games.

specifically: the tactic's rating is adjusted based on the rating of those who solve it, those who fail it, and in some way, the amount of time used to reach solution or failure point is also factored into the rating adjustment - - both for the user attempting it and for the tactic.

to say that a rating is not correctly rated is to ignore the fact that the rating on the tactic is as valid as any player's rating here - a rating is only an approximation of average ability - not an absolute.

for instance, someone who is familiar with bodens mate, and with smothered mate, may solve such in extremely short time, thus reducing the average time and the rating on the tactic, but may find anastasia's mate much more difficult to find and fail on it, thus raising the tactic's rating and lowering their own.

 

regards,

moderator

Atos

Anastasia is a Knight and Rook mate.

Jpatrick

I said before that the ratings distribution of the problems is multimodal. To see what I mean go to tactics trainer and then "view problems". This could be a symptom of trouble with rating algorithms or it could be a reflection of how material is selected.  Perhaps there are other explanations.

I have also observed before that problem Rd's are too high. Yeah, the problem ratings should float, but they should float in a narrow range once their proper level is found.  Their "skill" doesn't change, so their Rd value should theoretically converge to zero after enough valid attempts.

JillKovacs

I don't pretend to understand the algorithm. What I *am* saying is that something has changed. Before, no matter the problem, no matter how many moves, if I got all of the moves right, in half the average time, the score was 80%. Less than half the time, it was higher than 80%, no matter the actual *amount* of points added. Now, I have actually solved the problem in half the average time, and had the percentage be very low, and even had points subtracted. And then some problems seem to work the old way. Something changed.

JillKovacs

OK, above, everywhere I said half of average, change it to better than average. I just realized that the time counter on the bottom is double average time. I'm still maintaining that previously, on all problems, when I did the problem in average time, the score was 80%.

kittiehart

PLEASE HELP -- Does anyone know where on this site to find the listing of the perameters of how games here are scored?...e.g. how many points lost by timing-out vs. being checkmated?  TY!!