Tactics Trainer Problem Difficulty

I don't know what has changed. The problems are still great and I love the trainer, it's just my rating suddenly dropped 200-300 points in a few days. Maybe it's old age, but I suspect something changed in the algorithm. Maybe ratings were inflated and it was done as a correction. 

benkku52. I've noticed soemthing like that too. My TT rating is +/- 1300 and recently I've been getting mainly problems quite a bit below that with a few quite a bit above. Of course you don't know when you get the problem what the rating is, so you see a hanging piece and you don't know whether to grab it quick (because it's a low-rated problem) or whether the hanging piece is just a distraction and there's a subtle mate in 3 hidden in there somewhere. So it's hard to score decent points and at the same time, boy do you pay for it if you get one of the "easy" ones wrong! But I'm quite philosophical about it. In a game nobody tells you how deep you should be looking. Sometimes an easy opportunity pops up unexpectedly (at my level anyway) other times you search and search and you're lucky if you find anything.

The one enhancement that I think would make TT even better is to have a different scoring system for quickfire chess and for long format chess. The current system works well for bullet and blitz, I think, but it's a bit annoying for those of us who like to take our time. (So I focus more on my pass rate than my rating). My long format scoring system would have:

1) No minus points if you get it right, no matter how long you take (within reason, I guess).

2) No plus points if you get it wrong, even if you get 4 out of 5 moves or whatever, if you don't get it completely right then you haven't seen it. At best you get less of a minus.

I think that would be a better way for correspondence players and people who play standard chess with a lot of time on the clock.

I'm not saying that time doesn't come into it at all in the long format version.

If you get it 100% right very quickly you score maximum points. If you get it 100% right very slowly you score zero points.

If you get it wrong you score minus points. Less of a minus if you get some moves correct. Maximum minus if you take ages and still get the first move wrong.

There's still a premium for doing it fast but that premium never outweighs the primary object of getting it right.

@Josechu, it sounds to me like you do not know how the ratings are calculated in tactics trainer. If you score 50% against a problem with the same rating you get +-0 and that is the score you get for solving about 1,57 times the avg. time (not sure exactly, but a bit above 1,5).

http://www.chess.com/tactics/help#rating

I think I both agree and slightly disagree with benkku's reasoning. At first I was against solving puzzles against the clock, but now I think that can be a good thing. I don't believe in the reasoning with the 3 minutes/move though. The distribution of time for each move is very divided and you generally use more time in critical positions (and with good time management you will have enough time there, easier said than done though). Good tactics trainer problems only include critical positions (meaning one line is clearly best and not several winning or drawing lines), so comparing to otb games you probably don't need to move as fast and it could cause bad habits.

Another thing is that tactics trainer is too nice when your failing a problem half through. I think you should always score 0% when you didn't get all the moves right (You found the queen sac, but not the following mate, well done you get some rating points )

If I would decide things I would be more strict and only have 100% when succesfull and 0% when unsuccesfull. You get a given time for a problem and score 0% if you don't make it within the time (but you may still solve it afterwards if you want anyway). There would be 4 modes "fast", "normal", "slow" and "timeless". I'm not sure about specifics, but the idea is that the timer will be based on the avg.time for each problem and different on different modes and the timer will not be there in "timeless mode".

These things are slightly off the issue here though, it's more important that people don't get a lot of lower rated problems where it will be very hard to maintain a rating. If there have been a deflation/inflation in tactics trainer problems I don't see it as a big issue (as long as it's not a huge difference) and I don't see it as something that should be attempted to fix.

Martin. I know how it works currently, I've looked into it quite a lot. What I'm suggesting is an alternative system that is not so sensitive to time and where you do not get a minus score for solving in > 1.57 times the average time (but only if you opt for the "slow" algorithm. If you prefer the current "fast" option then you can opt for that.)

In my "slow" system if you correctly solve a problem with a rating = to your rating, and you are the slowest candidate ever to solve it correctly, then you score zero. If you are very slow but not quite the slowest then maybe you score 1. If you are the fastest to solve it correctly you score maximum points. See what I mean?

There are some wrinkles of course but the basic principles are described in my post #24 above.

Martin - my post crossed with your post #28. Looks like we basically agree.

@Josechu, Sorry I missed post #24 for some reason and thought you tried to explain how it currently works in post #26. My bad. I guess we basically agree indeed.

Mine is higher than my rating, though. Probably the randomness isn't working that well, OR that there are'nt enough high rated problems for you!

Recent Problems

I think it's important the problems you get are within 100 points range. That seems to be the case for cshuenss, but we have seen others with a higher rating difference. There could indeed be something wrong with the random algorith.

I collect my TT data because I like to do my own analysis. Graph below shows the average problem rating of the problems I've attempted, by month, against my average rating. For ages the two lines are indistinguishable. But from 2013, roughly they have started to diverge. I assumed it was because I've done so many problems now, within quite a limited range, that there are no problems left in my range so they give me some higher and some lower, and they don't quite get it right. I may try resetting my stats just to see what happens. But like I said above, it doesn't bother me that much. It amuses me that I seem to do better on the tough problems than the easy ones!

we are going to tweak this algorithm again to make it better!

I am not sure why you failed the one problem that is nearly 1000 points below your rating. It could have been a touch screen issue. Phones suck for that. I won't even try the TT on my phone because of failing, due to it.

I often feel that the system for which they are established in difficulty by rating is severly flawed.

If you failed it because, you simply had a different and most likely, completely winning idea, that was .001 of a point below the computer's assessment, especially, if it was a 7 move puzzle, you shouldn't get dinged for it in my opinion, to begin with. I get frustrated when I see 2 move puzzles that I can solve in 3 tries that are rated 2000+, but 5 to 7 move puzzles at my rating range, that take 5 to 10 tries. That is pretty good evidence to suggest that we get dinged for choosing lines that are winning when we shouldn't and that the system for predetermining difficulty is flawed.

I propose a system that rewards you according to your true ability, as opposed to how often you can match a specific engine's top choice for a move. My goal isn't match Houdini move for move. The last time I checked, that gets you banned. I should instead be rewarded for chosing winning lines. That is the goal of any chess player, at any level.

Erik,

Thanks for looking at this to improve the algorithm.  My main complaint is that for the past 2-3 weeks, I am only getting problems that are 100-250 points below my current rating (around 1700), which makes it very difficult to improve my rating.  My rating had consistently been between 2200-2400, then three days in a row I only got problems rated between 1600-1700, which caused a big drop.  All I want is problems that are consistent with my rating, which will help me gain rating points and get better problems.  I hope to see a change soon.

You don't need to speculate too much why I failed a puzzle. It was a good puzzle and I simply don't manage to solve all problems rated that low. I'm not that consistent at solving lower rated puzzles. I just couldn't see a knight fork for some reason.

I'm assuming your referring to the search algorithm when starting a new puzzle. In that case you could also keep in mind that some people get a lot of lag when they have made a lot of puzzles (which they solved by using reset history, if I remember an old forum I cannot find correctly)

Thanks for looking into this.

If that is the case, it lends even more creedance to what I am saying. Master level players shouldn't make mistakes that that sub 1200 players make...I realize even GM's make silly mistakes from time to time. If a puzzle is really worthy of that rating in my opinion, you don't make that mistake.

Having said all of this, considering I am not a computer programmer, I am not sure how to incorporate a preattempted rating system for puzzles, or a way to integrate a system for rewarding a player for choosing winning lines. This could be done on a basis of points rewarded for how close to the best and all of the way down to a drawing line, for which you would be rewarded nothing. When you play losing moves, that is when you should get dinged. Also, the system should be set up so that you can chose it to reward or penalize according to the type of chess you play and the difficulty of the problem. Three out of the 10 people who try a puzzle might have gotten lucky and picked the right move quickly, driving the average solved time down, too low, while 7 people might have failed it. My skill shouldn't be required to keep pace with the luck of others.

My tactics tainer rating was 2500-2600 for almost 4 years then fell suddenly to the 2300 range. While disappointing, it better represents my Fide rating (2390). It might be that once you fall out of a range, it's very hard to get back up to that level unless you work very hard and don't cut corners. This includes doing too many problems at once, doing problems sleep deprived, or playing the move that "looks right" without calculation. 

Let's make it happen!

Well, having a tactics rating of 2000+ doesn't makes me a master level player.

There are some bad problems with several winning lines, but those are just bad problems that should be removed/changed (since alternative winning lines currently isn't an option). It shouldn't matter how close you are from solving a puzzle, a failed puzzle is a failed puzzle. I don't care weather you turn a win into a draw or a win into a loss, both should result in a failed puzzle.