The problem with tactics trainer

Because there is such a heavy dependence on time in tactics trainer, many of the answers submitted by a user are, in fact, guesses.  I know that I'm going to lose points once I see 3/4 of the bar go away.  As a result, when I get close to that level, I take my best guess, whether or not I've actually worked out a solution.

This really skews the ratings of problems and, worse yet, skews the time required.  If I get it right, it seems likely that the algorithm will assume that it only took that amount of time to do it.  If I get it wrong, I'm not sure.  I would guess the problem rating goes up, but the time required might not.  After all, I finished in the required amount of time, so it might assume there is no need to add time.

At any rate, I would simply ask that you consider that the method used results in a whole lot of guesses, which lowers the "training" quality, and skews the assumptions that go into assigning both ratings and time required.

do you have a better suggestion?

Hi dpruess,

On Chess Tempo, the untimed rated mode is far more popular than the timed rated mode (although the untimed mode is enabled by default, so this statistic may be somewhat skewed).

In any case, I personally far prefer the untimed mode, as IMO there are major benefits (for both pattern learning and visualisation practice) in being able to spend as much time as you like on a problem.

I know that chess.com premium members can do untimed unrated problems, but IMO there is far less motivation involved when there's no rating change at the end.

Regards,
Tom

hmm, so adding a hybrid mode. interesting...

I would support the above...

I think that for my learning (and for others?) the first step is prob. to learn the tactics - the ability to then tackle them at speed is a secondary step...

so would love the ability to switch

Alasdair

My suggestion is that you should not get negative points when you find the right answer. Let the time affect the points from zero to maximum, so that it is still a factor. But the current system does encourage guessing because if I take the time to work out the full solution I will lose points. Oftentimes I have found a good answer, not the best answer, so I rush to make that move so I won't lose points on time.

That would be an interesting feature, though I have changed my style to ignore the time now. The point is to solve the problem and if I can't do it in the given time, then I lose some points. I'll try to do better next time.

I do agree you probably shouldn't lose points for  successfully completing the puzzle, even if you don't do it in time. Though, I understand the rationale.

What I would find more useful, is a hybridization of the rated and unrated modes so that you can use features of either. I would like to be able to set a ratings range and still get the benefits of the tracking. It would require a revamping of the rating formula, so that low rated puzzles wouldn't give points if the range fiffernce was too great (someone with a 1800 doing a 1400 wouldn't get points -- though the difference could be higher). I know there are some 1400 level puzzles that I just don't get and being able to get some in my problem ranges and tracked would be nice.

The other thought I have...

lets say a 2 move puzzle... at the moment, if you get the first move right, but not the second you get some credit.

however if you get the puzzle wrong, and then have another go and get it right you get nothing...

would it be worth having some value to having a second go and then getting it right? esp. where the move you first make isn't necessarily a bad / wrong move it is just not the best (the purpose of the puzzle...)

Alasdair

It would be interesting to know how the scoring exactly works. In particular, if we have a problem with say 5 moves deep solution and I get 3 of them correct, how well I score compared to the perfect solution. Maybe the answer is out there somewhere but unfortunately I could not find it.

Anyway, my gut feeling based on my short using experience is that partial solutions score too well. After all in a real game partial solutions usually bring you nothing but trouble. Therefore I think that also TT should award relatively little from partial solutions. This is especially true with the more advanced problems, since surely exact calculation of variations is a necessity for any strong player.

Another thing is that in some problems it's too easy to guess the right answer. This can happen if the solution sequence stops too early because from certain point on there are several equivalently good solutions and TT currently doesn't support multiple solutions. See for example http://www.chess.com/tactics/server.html?id=60301 where the solution is only one move long altough in reality you have to calculate deeper to chek it really works and also to see the real tactical point of the whole exercise. Personally I think that such problems should be discarded altogether.

There are a few things that could be done to minimize the problem.  Most have been suggested.  Changing the scoring system so that you don't lose points for right answers is a good suggestion.  Frankly, I can't imagine the down side of such a system for a training aide.  The untimed, but rated, mode is also a good idea.  Another option is an all or nothing, constant timed problem.  In other words, every problem gets 3 minutes (or 3 minutes for the first move plus 20 seconds for each additional move?)  If you get it right within three minutes, it counts as a win, and ratings are adjusted accordingly.  In this system, there would still be some guessing, but not as much.  What I particularly dislike in TT is that I see what I think is probably the best move, but the problem only allows 36 seconds, so I don't sit and work out the problem.  I go with my first instinct, because I don't have time to work out the problem.  Sometimes I'm right.  Sometimes, I don't spot the bishop on the other side of the board guarding the key square.  I think ratings could still be meaningful if there was a time limit, and I could work out the answer to be sure it really was the right answer.  Meanwhile, for more difficult problems, I might never spot the answer.

There's a fairly simple test to see if I'm right about the number of people guessing, and the way that skews the ratings.  Sample the database of problems and look at the history of the problems.  If I'm right, there will be a general trend of problem times getting shorter as time goes on.  That shouldn't happen.  The time required to solve a problem should be pretty much constant, but I think that all the guesses will cause compression of the time required.

ETA:  And it's not as if the solutions are mutually exclusive.  The same problem could have a "three minute" rating, and a "variable timed" (i.e. like today's system) rating.  Both would be valid.

What bugs me a little:

Some failures aren't as bad as others.

Eg, you spot a mate in 3, and get no points for the 1st move, because there's a mate in 2 .Somebody else flukes the 1st move, but fails to find the correct follow up, and actually gains a point or 2.

I've never heard of the "Adaptive tactics server."