Periodic drop of rating in Tactics Trainer ...
Sort:
May 20, 2008
0
#3
I would also like to add that it would be nice if your rating gained some stability after playing a lot of puzzles. If I have no history in Tactics Trainer, I can understand the rating adjusting by large amounts initially, but after a certain point the system should have learned enough about my skills to remove some of the volatility of the ratings adjustments (for me and the puzzle). If I have consistently been solving 1400-1600s, and goof a couple times on some 1200 puzzles (sometimes because I am looking for a more complicated tactic thinking it is in the 16-18 range), it doesn't make sense for me to drop by 200 points.
Today was an outstanding day. Not less than 6 times my rating suddenly became 600-900 from 1100 - 1500. Interval is about 10-20 problems. Most often it happened when problem is partially solved of failed. Only one time it was after solved problem. It looks like next problem becomes very easy and you notice that your rating now is 800(+60) while the problem before was 1300(+10). Remember the game "Prince of Persia"? It is very similar to behaviour of tactics trainer - fall in the middle - start from the beginning ;)
I thing that "twin peaks" of player's rating distribution in tacktics trainer are not natural and represent the bug.
May 20, 2008
0
#5
Yes, I agree. I am a slow thinker and need time to absorb the board. I have passed one and lost points. Then I made mistakes in several because I tried to stay on time. My purported rating is lower than when I was ten years old! Hmmmmm.
darius, I talked not about loosing the points because the time is over. 99% of my attempts are in time.
I haven't a problem. I solve 3 problems per day, it's enoug, and my rtg fluctuate between 2150 and 2350. I think it's OK. Sometimes I lose points even when I solve a problem, for I was slow; I think it's OK too: to be fast, is a part of a training! My passed rate is 57% - without a time limitation, and a posibility to use an analysis board, it would be near to 100%.
Sometames an estimated rating for a problem is strange - to high or to small.
There are mistakes sometimes, and it's more frustrating; however, not so often and it can be fixed. Last week I found that one problem has a double solution, and the staff wipe it out after my warning.
@erik: the staff didn't answer my warning, 4 days ago, about the problem 0020924! There are no a solution! And I lost 47 points on it.
May 20, 2008
0
#8
The way our rating system works on Tactics Trainer is we did leave a lot of volatility in the formula since its somewhat nice to be able to gain lots of rating points in a single day without having to go through dozens of tactics.
However, the way our timer works is broken down below. These are multipliers we apply to your final score based on where you finished:
>= 2x Standard Deviation = 100%
>= 1x Standard Deviation = 90%
>= Average Score = 80%
>= -1x Standard Deviation = 60%
>= -2x Standard Deviation = 40%
>= Timeout = 20%
So once you get about 1.5 Standard Deviations below the average time on a problem, you will lose rating points even if you solve. If you solve the problem and have no time left at all, you get a 20% score.
Our standard deviations are defined as:
2 SDs = 90%
1 SD = 75%
Avg = 50%
1 SD Below = 25%
2 SD below = 10%
Hopefully this makes more sense now to everyone. :)
If we just allowed everyone to pass regardless of the timer, than our problems would suffer a serious deflation of rating points.
Jay, your post is not an answer on my problem. Can you imagine the situation when you get -500 rating decrease after _one_ problem in case that your rating and problem's rating were very similar (for example 1300 and 1200)?
May 20, 2008
0
#10
No, that sounds impossible to me Jippo. :) On your tactics home page we show how many points your rating was adjusted for each problem solved/failed. I don't see anything here that indicates it changed 500 points. Next time it happens, can you please stop immediately what your doing and post here and let me know that it just happened.
May 20, 2008
0
#11
In fact, here is a copy paste of our log files of the last ~50 tactics problems you have attempted. It goes in order of date, with your last attempt being the last line at the bottom. The way to read this is on each line, the data consists of:
User:<user_id> (<old_rating>|<old_rd> -> <rating_change>|<new_rd>) vs Problem:<problem_id> (<old_problem_rating>|<old_problem_rd> -> <problem_rating_change>|<problem_new_rd>) -> <your_score>
User:905111 (1614|132.07 -> -11|131.38) vs Problem:29384 (1685|135.54 -> 12|134.06) -> 0.28282828282828
User:905111 (1603|131.38 -> 38|130.66) vs Problem:23410 (1563|135.11 -> -40|133.52) -> 0.97755775577558
User:905111 (1641|130.66 -> -26|131.45) vs Problem:25389 (1813|143.97 -> 30|142) -> 0
User:905111 (1615|131.45 -> -30|131.41) vs Problem:31610 (1746|131.84 -> 30|131.71) -> 0
User:905111 (1585|131.41 -> 43|130.73) vs Problem:31137 (1642|133.4 -> -44|132.27) -> 0.90793650793651
User:905111 (1628|130.73 -> 38|130.38) vs Problem:27709 (1713|132.75 -> -39|131.95) -> 0.81538461538462
User:905111 (1666|130.38 -> -1|130.36) vs Problem:25206 (1779|132.84 -> 1|132.29) -> 0.44064171122995
User:905111 (1665|130.36 -> 49|130.25) vs Problem:27566 (1769|132.27 -> -50|131.74) -> 0.92083333333333
User:905111 (1714|130.25 -> 9|131.99) vs Problem:29608 (1474|131.49 -> -9|133.01) -> 0.87272727272727
User:905111 (1723|131.99 -> 28|131.12) vs Problem:23619 (1676|132.28 -> -28|131.34) -> 0.87567567567568
User:905111 (1751|131.12 -> -43|134.48) vs Problem:30490 (1600|350 -> 259|268.17) -> 0
User:905111 (1708|134.48 -> -23|135.32) vs Problem:20833 (1925|150.6 -> 28|148.25) -> 0
User:905111 (1685|135.32 -> 1|133.91) vs Problem:28535 (1606|129.98 -> -1|129.78) -> 0.54623655913978
User:905111 (1686|133.91 -> -20|132.74) vs Problem:22579 (1736|137.3 -> 21|135.33) -> 0.2140625
User:905111 (1666|132.74 -> -21|134.12) vs Problem:21760 (1901|146.62 -> 25|145.39) -> 0
User:905111 (1645|134.12 -> -29|134.16) vs Problem:21515 (1798|154.1 -> 36|149.69) -> 0
User:905111 (1616|134.16 -> -21|135.43) vs Problem:22501 (1863|140.15 -> 22|140.34) -> 0
User:905111 (1595|135.43 -> 36|133.83) vs Problem:22132 (1637|133.95 -> -35|132.7) -> 0.83076923076923
User:905111 (1631|133.83 -> 49|133.6) vs Problem:29200 (1783|133.11 -> -48|133.03) -> 0.83571428571429
User:905111 (1680|133.6 -> -1|134.12) vs Problem:30854 (1877|136.22 -> 1|136.22) -> 0.28385744234801
User:905111 (1679|134.12 -> 4|134.05) vs Problem:27367 (1522|143.37 -> -4|141.32) -> 0.73333333333333
User:905111 (1683|134.05 -> 50|135.07) vs Problem:24627 (1918|133.27 -> -50|134.43) -> 0.75714285714286
User:905111 (1733|135.07 -> -29|134.76) vs Problem:25561 (1886|143.27 -> 32|141.19) -> 0
User:905111 (1704|134.76 -> -49|133.22) vs Problem:29199 (1682|133.18 -> 48|132.01) -> 0
User:905111 (1655|133.22 -> -51|132.09) vs Problem:26854 (1610|133.05 -> 51|131.96) -> 0
User:905111 (1604|132.09 -> 48|132.33) vs Problem:30635 (1767|131.7 -> -47|132.02) -> 0.81971830985915
User:905111 (1652|132.33 -> 1|132.52) vs Problem:21806 (1490|132.32 -> -1|132.51) -> 0.61347150259067
User:905111 (1653|132.52 -> -32|132.14) vs Problem:21396 (1771|133.54 -> 32|132.93) -> 0
User:905111 (1621|132.14 -> -40|131.25) vs Problem:26985 (1664|133.87 -> 41|132.58) -> 0
User:905111 (1581|131.25 -> 37|132.29) vs Problem:28047 (1546|227.35 -> -94|198.42) -> 1
User:905111 (1618|132.29 -> -30|132.35) vs Problem:21677 (1757|143.77 -> 34|141.31) -> 0
User:905111 (1588|132.35 -> 41|133.27) vs Problem:25786 (1797|134.86 -> -42|135.3) -> 0.69
User:905111 (1629|133.27 -> -34|132.58) vs Problem:29150 (1731|134.53 -> 34|133.56) -> 0
User:905111 (1595|132.58 -> -23|133.72) vs Problem:27895 (1813|146.87 -> 27|145.24) -> 0
User:905111 (1572|133.72 -> -5|132.38) vs Problem:29850 (1551|131.9 -> 5|130.98) -> 0.47536231884058
User:905111 (1567|132.38 -> -23|133.29) vs Problem:31560 (1777|133.19 -> 23|133.94) -> 0
User:905111 (1544|133.29 -> -1|133.98) vs Problem:29007 (1750|134.09 -> 1|134.63) -> 0.32691867124857
User:905111 (1543|133.98 -> -7|134) vs Problem:27966 (1712|135.97 -> 7|135.58) -> 0.2225641025641
User:905111 (1536|134 -> -1|134.11) vs Problem:25209 (1708|139.37 -> 1|138.37) -> 0.31475694444444
User:905111 (1535|134.11 -> -31|133.6) vs Problem:22264 (1667|135.18 -> 32|134.43) -> 0
User:905111 (1504|133.6 -> 12|133.8) vs Problem:28323 (1322|129.06 -> -12|130.16) -> 0.85333333333333
User:905111 (1516|133.8 -> -30|133.47) vs Problem:27482 (1655|137.22 -> 32|136.15) -> 0
User:905111 (1486|133.47 -> 23|134.2) vs Problem:26710 (1274|131.58 -> -22|132.67) -> 0.99259259259259
User:905111 (1509|134.2 -> -38|133.02) vs Problem:28451 (1576|134.86 -> 38|133.53) -> 0
User:905111 (1471|133.02 -> -23|134.64) vs Problem:28785 (1260|201.84 -> 53|187.35) -> 0.46315789473684
User:905111 (1448|134.64 -> -23|135.21) vs Problem:27989 (1664|130.19 -> 22|131.59) -> 0
User:905111 (1425|135.21 -> 26|134.26) vs Problem:28569 (1541|133.78 -> -26|133.15) -> 0.63040935672515
User:905111 (1451|134.26 -> 16|133.71) vs Problem:30721 (1313|129.46 -> -15|129.92) -> 0.84736842105263
User:905111 (1467|133.71 -> -24|134.33) vs Problem:28546 (1669|139.83 -> 26|139.24) -> 0
User:905111 (1443|134.33 -> -4|135.13) vs Problem:30552 (1666|135.5 -> 4|136.08) -> 0.2
User:905111 (1439|135.13 -> 48|133.54) vs Problem:28033 (1450|135.71 -> -48|133.98) -> 1
User:905111 (1487|133.54 -> 52|133.54) vs Problem:22268 (1653|131.31 -> -51|131.77) -> 0.856
User:905111 (1539|133.54 -> -24|134.11) vs Problem:22237 (1740|134.32 -> 25|134.74) -> 0
User:905111 (1515|134.11 -> 61|134.33) vs Problem:26678 (1702|131.95 -> -60|132.6) -> 0.92482269503546
User:905111 (1576|134.33 -> -26|134.6) vs Problem:25825 (1764|138.39 -> 27|137.84) -> 0
User:905111 (1550|134.6 -> -33|133.84) vs Problem:30506 (1670|134.7 -> 33|133.91) -> 0
User:905111 (1517|133.84 -> 22|132.8) vs Problem:27937 (1436|131.95 -> -22|131.33) -> 0.84705882352941
User:905111 (1539|132.8 -> 54|134.08) vs Problem:28319 (1775|135.68 -> -56|136.43) -> 0.8
User:905111 (1593|134.08 -> 62|135.36) vs Problem:27051 (1843|134.92 -> -63|136.06) -> 0.86299212598425
Jay, I caught the situation. This time I wrote down each current rating value and recognize the error that is much less than before, but it is dafinately a bug.
most recent log is .... 1368(-51), 1405(+37), 1378(-27), 1266 (+63)!
As you see before the last problem the recorded rating was 1378 but actual was already 1203. The difference is not 500 but 175 is also not a zero ;)
Now I stop my activity in tactics trainer before your responce.
P.S. I think that additional column "player's rating" in "Recent Problems" table will be useful.
P.P.S It seems that most critical interval is about 1000-1300. The only once this bug occurred at rating 1500.
May 21, 2008
0
#14
Ok, I figured it out. It's because you are skipping TONS of problems. Every time you skip a problem, a few minutes later a "cron script" runs on our servers and marks those skipped problems as failed, so you have a huge rating drop when that happens. Here is output from our skipped problem log. Your user id is 905111, look at all of your entries!
--------- Wed, 21 May 2008 08:15:00 -0700--------
User 905111 - Problem 23599
User 905111 - Problem 26929
User 905111 - Problem 24730
User 1368182 - Problem 28819
User 905111 - Problem 29898
User 905111 - Problem 32035
User 905111 - Problem 23263
User 905111 - Problem 28423
User 905111 - Problem 26359
User 905111 - Problem 31221
User 1368178 - Problem 22027
User 905111 - Problem 23575
--------- Wed, 21 May 2008 08:30:00 -0700--------
User 905111 - Problem 29470
User 1373310 - Problem 815
User 905111 - Problem 24732
User 905111 - Problem 30051
User 905111 - Problem 21693
User 1382966 - Problem 26250
User 905111 - Problem 30853
User 905111 - Problem 29336
User 666811 - Problem 26187
User 905111 - Problem 27573
User 905111 - Problem 27820
User 905111 - Problem 21625
User 905111 - Problem 32015
--------- Wed, 21 May 2008 08:45:00 -0700--------
User 905111 - Problem 32532
User 905111 - Problem 28781
User 1336508 - Problem 29929
User 905111 - Problem 32202
User 905111 - Problem 31310
User 905111 - Problem 28163
User 905111 - Problem 29127
User 1387193 - Problem 26226
User 905111 - Problem 31125
User 905111 - Problem 29162
--------- Wed, 21 May 2008 09:00:00 -0700--------
User 905111 - Problem 27423
User 905111 - Problem 31139
User 905111 - Problem 30007
User 905111 - Problem 23584
User 905111 - Problem 27365
User 905111 - Problem 29947
User 905111 - Problem 26486
User 905111 - Problem 31644
User 905111 - Problem 29510
User 905111 - Problem 31587
User 905111 - Problem 31638
User 1388102 - Problem 27709
User 905111 - Problem 29960
May 21, 2008
0
#15
I have added a new "My Rating" column to the recent problems chart...but you won't see it showup until after our next release, which will probably taken place early next week.
How i could skip this problems? I try to solve each problem that I receive.
Now I see in list of my recent problems the tons of problems that I didn't even see? Some of them are failed, others are passed. But I stoped the activity as I mentioned! What's going on? My current rating becomes 1025 without any influence from me ;)
May 21, 2008
0
#17
If you refresh that tactics trainer window, it is the same as skipping the problem. Don't ever refresh it or push ctrl-r or f5 or anything like that. Only use the buttons in the window. I'm guessing you are refreshing the window or something.
Today, I did refresh this window only once when the problem "hangs". I used only buttons in the window and even didn't switch windows for a long period. After I caught the problem I immediately close this window and post report here. (I did not restart browser after that). But you say (and I see it in my log) that after that very many problems was failed or even passed and my rating becomes even smaller for the period of my inactivity. It seems that you do not believe that there is a bug and suppose that I'm playing with controls. it's ridiculous.
Are you interested to find the problem? The problem is not me ;)
Yesterday I did not refresh the tactics trainer window too.
May 21, 2008
0
#20
Also, be sure you aren't ever closing the tactics trainer window AFTER a problem has started.
Well, you can see the logs I sent you, you are the ONLY user out of thousands using the tactics trainer that seems to be having this problem...so yes, my immediate thought is that you are skipping problems somehow. :)
... is really annoying.
When the rating decreases on 300-500 points without any reason several times a day it is not fun. Often but not always this occures when problem was failed (and indicated rating loss is much smaller than even 100). One time it was when problem was passed.