Major Proposed Change: Ratings Cap Above 2400 for Unverified Players

Sort:
Avatar of sapientdust
WIshbringer wrote:

The Tactics Test is an interesting concept, but hard to put into practice. If you are going to have an instrument with such a high ceiling, you are going to have to find 2400+ players willing to help develop it. In fact, at that level, you might as well include items to assess positional understanding as well as sophisticated endgame knowledge. If you see where I'm going with this, the instrument, whatever its final form, will be time-consuming, and potentially expensive to construct. Even if a reliable and valid measure of this sort can be developed, I am not certain it will be worth the effort.


This criticism doesn't make any sense to me, because you're talking about something totally different than I'm talking about. You don't have to "develop" any instrument. You just have to time how long each tactics problem takes to be solved, and after a month or two, you create a new library of problems that were solved in 6 seconds or less by a significant number of players. Then, you can analyze the average rating of the players that solved those problems in 6 seconds or less.

There is no reason to add positional understanding or endgame knowledge. The purpose is to detect cheaters, not to exhaustively analyze every facet of a chess player's strength.

I'm still waiting to hear a single reasonable criticism of why tactics training using only problems that can be solved in seconds is not a reasonable way of detecting cheaters -- or at least identifying people who are extremely likely to be cheaters.

Avatar of bigpoison

Though I don't know it, Mouse, I would assume that the "premium only" stat you've displayed at the bottom isn't calculated in just the last 90 days, but instead, over the duration of your games.

I could be wrong, though.  It happens so often.

Avatar of pathfinder416
sapientdust wrote:

I'm still waiting to hear a single reasonable criticism of why tactics training using only problems that can be solved in seconds is not a reasonable way of detecting cheaters -- or at least identifying people who are extremely likely to be cheaters.


I can see the argument for blitz/bullet ratings, but players who rarely enter the speed world shouldn't be tested in that way. It demands they be good at something they don't practice.

Avatar of woton
rooperi wrote:

I think it's a little sloppy that a graph and a table representing thew same thing are so different, this should be fixed....


 It's not unusual for there to be a difference between tabular data and graphical data.  Not all points are plotted, and often graphs are "smoothed out" for a pleasing visual effect.

Avatar of dpruess

the change will affect all rating pools. if the bullet pool is currently inflated, we can take measures to deflate it (bringing everyone's rating down 200 points or whatever is necessary), and thus this will affect fewer people.

sapient how do you administer this tactics quiz? how do you track the time someone takes on every problem?

why 2400: it's a good representation of how high a non-titled player could possibly get without cheating; of course there are exceptions, but that's how you guarantee that this is only affecting a handful of players. it's also hopefully low enough that being cut off at that point crimps the prestige gains of cheaters sufficiently to deincentivize them. i think 2300 could have been a good number too.

(i'll repeat that the bullet pool may need a deflationary adjustment)

Avatar of dpruess

it is possible to cheat in bullet, and people do.

Avatar of sapientdust
pathfinder416 wrote:
sapientdust wrote:

I'm still waiting to hear a single reasonable criticism of why tactics training using only problems that can be solved in seconds is not a reasonable way of detecting cheaters -- or at least identifying people who are extremely likely to be cheaters.

 


I can see the argument for blitz/bullet ratings, but players who rarely enter the speed world shouldn't be tested in that way. It demands they be good at something they don't practice.


It doesn't require practice. You just can't improve at chess without improving your tactical ability, and as your tactical ability improves, the tactics that leap out at you instantly and require no thought to notice (because they are patterns that you recognize) increases.

Avatar of dunce

Just a point in passing. It takes some time to get used to the problem-style of the Tactics Trainer program. I do chess problems on another site, and their style of problems is completely different.

Avatar of sapientdust
dpruess wrote:

sapient how do you administer this tactics quiz? how do you track the time someone takes on every problem?


The time for each problem can be tracked in a similar way as you already track how long it takes to solve tactics problems. Each individual problem would be timed alone, just like you do now, and like they do on the chess tactics server (which also is mostly problems that you recognize instantly and can solve in a few seconds, fyi, unlike the problems here and at most other tactics sites that i've seen).

I'm not sure what you mean by administer. You could just randomly force them to do 5 or 10 tactics problems every now and then when they start getting close to 2400, perhaps randomly after a game finishes -- maybe allowing them a certain number of byes when they can opt to do the problems at a later time, but at some point requiring them to do them when asked.

Avatar of Martin_Stahl
Reb wrote:
Martin_Stahl wrote:
Reb wrote:

 an online rating of +300 compared to one's otb isnt alarming but when it becomes +500 and more its very suspicious/alarming and +1000 and more is a clear/flagrant cheat...... 


Do you consider this only true for highly rated players here? I'm ~700 point higher in correspondence here than my OTB rating. While I can point to a number of contributing factors for that, cheating isn't one of them


Yes, imo its suspicious. 


OTB and correspondence are much different. I can't use opening books and databases in OTB tournaments. Can't play with the analysis board there either. Don't get a whole lot of people timing out on their games in tournaments.

At our local club, I have been performing very well over the past few months in casual games with a player in the mid 1600-s OTB. I don't have any of the games moved over to my database yet from this year but I'm pretty sure I'm scoring very close to 50% of my games won against him, with no game-time resources.

My performance at tournaments is spotty and that shows in my OTB rating. I guess you have the right to be suspicious of whoever you want; I can just guarantee that I always play by the rules.

Correspondence is a different beast, especially when it comes to ratings and gameplay.  Shoot, if you have a low enough average opponent rating you can still get pretty high here. I looked at a player with a 2350+ rating and the average opponent is pretty low, with a vast majority under 1600; only one over 2000 win. It seems to me the rating system itself can get pretty out of whack, especially if you play a lot of games against lower rated opponents.

Avatar of sapientdust
dunce wrote:

Just a point in passing. It takes some time to get used to the problem-style of the Tactics Trainer program. I do chess problems on another site, and their style of problems is completely different.


Yeah, that's true, but I think it's less true of the type of problems I'm thinking about (see chess.emrald.net for example). The type of problems that most players recognize the pattern for and can solve in seconds are different than the problems that you have to think about carefully and consider different variations and possible solutions for. I think the instantly recognized type of tactics problems require less getting used to than other types of problems, once you realize that the problems ar of that kind.

Avatar of dunce

Another thought: If there are only a few hundred players in question and your turnover is low, it should be possible to use a whitelist/probation system (in addition to whatever other systems you want to use).

I still think random testing of the new (non-whitelisted) 2400+ players' games would be a powerful tool.

Avatar of pathfinder416
sapientdust wrote:
pathfinder416 wrote:
sapientdust wrote:

I'm still waiting to hear a single reasonable criticism of why tactics training using only problems that can be solved in seconds is not a reasonable way of detecting cheaters -- or at least identifying people who are extremely likely to be cheaters.

 


I can see the argument for blitz/bullet ratings, but players who rarely enter the speed world shouldn't be tested in that way. It demands they be good at something they don't practice.


It doesn't require practice. You just can't improve at chess without improving your tactical ability, and as your tactical ability improves, the tactics that leap out at you instantly and require no thought to notice (because they are patterns that you recognize) increases.


I strongly disagree. If you pit a pool of blitz-seasoned experts against a pool of never-blitzed masters, overall the experts will win in a landslide. I say this because I've seen it again and again and again.

Avatar of sapientdust
RainbowRising wrote:
pathfinder416 wrote:
sapientdust wrote:

I'm still waiting to hear a single reasonable criticism of why tactics training using only problems that can be solved in seconds is not a reasonable way of detecting cheaters -- or at least identifying people who are extremely likely to be cheaters.

 


I can see the argument for blitz/bullet ratings, but players who rarely enter the speed world shouldn't be tested in that way. It demands they be good at something they don't practice.


Engines solves most TT problems id imagine.


Yes, as Fezzik noted, the key point is not TT problems, it is TT problems of the type that you recognize in seconds and can solve in seconds. If you limit the problems to 5 seconds to view and one second to move, and you ensure there are always a certain minimum number of pieces on the board, you almost entirely eliminate the possibility of somebody being able to input that position into their computer and respond within the required 6 seconds.

Avatar of sapientdust
pathfinder416 wrote:
sapientdust wrote:
pathfinder416 wrote:
sapientdust wrote:

I'm still waiting to hear a single reasonable criticism of why tactics training using only problems that can be solved in seconds is not a reasonable way of detecting cheaters -- or at least identifying people who are extremely likely to be cheaters.

 


I can see the argument for blitz/bullet ratings, but players who rarely enter the speed world shouldn't be tested in that way. It demands they be good at something they don't practice.


It doesn't require practice. You just can't improve at chess without improving your tactical ability, and as your tactical ability improves, the tactics that leap out at you instantly and require no thought to notice (because they are patterns that you recognize) increases.


I strongly disagree. If you pit a pool of blitz-seasoned experts against a pool of never-blitzed masters, overall the experts will win in a landslide. I say this because I've seen it again and again and again.


True but irrelevant. The question is whether there are people rated 2400 and above who have the tactical ability of players rated 2000 and below for the class of problems that can be solved instantly. We're essentially talking about a pattern recognition test, and the question is whether somebody who plays blitz and is rated 2000 or below knows as many patterns as somebody who doesn't play blitz and is rated 2400 or above. There is definitely some variety, but I think on average, there are statistically significant differences in ability to instantly recognize standard patterns that still hold true regardless of how much blitz or bullet one plays.

Avatar of pathfinder416
sapientdust wrote:
pathfinder416 wrote:
sapientdust wrote:
pathfinder416 wrote:
sapientdust wrote:

I'm still waiting to hear a single reasonable criticism of why tactics training using only problems that can be solved in seconds is not a reasonable way of detecting cheaters -- or at least identifying people who are extremely likely to be cheaters.

 


I can see the argument for blitz/bullet ratings, but players who rarely enter the speed world shouldn't be tested in that way. It demands they be good at something they don't practice.


It doesn't require practice. You just can't improve at chess without improving your tactical ability, and as your tactical ability improves, the tactics that leap out at you instantly and require no thought to notice (because they are patterns that you recognize) increases.


I strongly disagree. If you pit a pool of blitz-seasoned experts against a pool of never-blitzed masters, overall the experts will win in a landslide. I say this because I've seen it again and again and again.


True but irrelevant. The question is whether there are people rated 2400 and above who have the tactical ability of players rated 2000 and below for the class of problems that can be solved instantly. We're essentially talking about a pattern recognition test, and the question is whether somebody who plays blitz and is rated 2000 or below knows as many patterns as somebody who doesn't play blitz and is rated 2400 or above. There is definitely some variety, but I think on average, there are statistically significant differences in ability to instantly recognize standard patterns that still hold true regardless of how much blitz or bullet one plays.


Okay, I'll give you a 'maybe' :). Not having seen the tool myself, it might be testing something more intrinsic than pure blitz skill and perhaps it is applicable.

Avatar of Lelldorion

Here's my initial opinion: Have the rating cap if neccesary however make it possible to apply to reach 2400+ e.g. by allowing your games to be scrutinised by staff/computer etc. Cheaters who apply would be caught out immediately, genuine players could surpass the 2400 mark. Payment should be about the features you receive rather than them being additional in order to improve your rating.Reading previous comments it seems there are very few players above 2400 so there wouldnt be much additional workload.

Avatar of blake78613

An artificial rating cap will lead to a deflation of rating points.  This will also affect premium players.  Although they it may be possible for them to achieve ratings of greater than 2400 it will be much harder for them to achieve.

Avatar of ivandh

Wow! I wasn't expecting my idea to get this much attention.

The simplest way to do this would be to grab stats for 2300+ players in tactics trainer, and when someone hits 2400 run them through a set of problems. It would be easy to use elementary statistics to compare the results and see if there is a sharp difference. Those people go on the short-list for investigation.

There are still many variables which make it too difficult to fully automate this process, but if you can cut down on the number of members to be investigated that's still going to save a bundle of time and money.

Avatar of dpruess

blake presumably you haven't read the full thread, as the following explanation has already been given several times. since this policy would be affecting the ratings of (example: online chess) about 100 people in a pool of 280 000, it would not have a meausrable deflationary effect.

also, as we've said, if the rating pool deflates, we monitor that, and we can nudge it up. similarly when it inflates, we can nudge it down. so that's not a source of concern for anyone.

This forum topic has been locked