Are Bot's ratings accurate?

Sort:
Shaun67isthebest

No, nowhere near. 

Shaun67isthebest

Same @chesshypermaster 

calbitt5750
My theory is that at least at my level (662 Rapid, mostly 30 minute games) my rating corresponds with double the ratings of the bots, I.e., I’ll beat a 1300 bot most of the times but by no means always.
AlphaOmega444

I am a 700 player but I have beat quite a few 1500 bots, so either I'm not a 700 or the bots are just bad. 

glueShoe

To assess the accuracy and/or ELO of chess<dot>com bots (or your own) refer to the accuracy assessed by the stockfish analysis engine. You may then use this conversion table to map Accuracy to ELO. https://pastebin.com/EBPYhhku

On average, I've found the chess<dot>com bots above 2000 to be ~100-150 ELO points higher rated on the site than the actual Accuracy to ELO conversion table indicates. For example, the 2200 bots play on average ~=2050-2100, 2450 bots ~=2300-2350, and Komodo23 (2700) ~=2500-2600.

However, my sample size is a rather small number of my games against the bots. Chess<dot>com could do such an investigation across a much larger data set and perhaps it converges toward the stated bot ratings respectively.

Moises_pasteis

recently i've been stuck on the 1000-1100 rating, and I win the 1800-1900 bots, so there it is

SteelieMD

Probably accurate IF you took them in a contest between each other. Most bots can confidently beat bots that are at least 100 points below them. The games take longer the lower the mutual ratings are (and if they are below 1000, oh boy if the 50-move rule didn't exist, they would play forever).

In context of their human opponents, they're inaccurate. I would say at least around 300-500 points less than they are rated. Their rating doesn't matter that much anyway until they are at least in the 2000s. They're tactically alert in the middlegame but even 1500-1600 bots can randomly blunder a piece to an obvious threat or play useless positional moves.

They're at their worst in the endgame though. The 1900-rated Manuel couldn't convert a K+Q vs K endgame. A lot of them blunder drawn endgames that beginners would have no trouble holding, from just throwing away pawns to making insane moves.

Even the Maximum bot (3200-rated) isn't as strong as its rating suggests, having a relatively equal score with engines rated around 2800-2900 on the CCRL site.

tl;dr Mutually among themselves their rating is accurate. In comparison to human rated opponents, not so much.

Conurechess
A Bots rating based on how fast and accurate they can move not like a humans
GregSea

Yes, they are overrated, go kamikaze when clearly losing, and cannot handle endgames. Thanks for confirming.

But they're nice at forcing you to try new openings !

mpaetz

     There is no way to make bots play like actual 800 or 1500 or 2200 human players. The computer programs can easily outplay humans. What the programmers do is handicap the bot by limiting the depth to which they search, limit access to the opening database, and putting in occasional random moves (which will usually turn out to be blunders) in order to give the bots the desired practical strength. Even though the machines are much stronger, enough monkey wrenches are thrown into the works to cause enough losses to give the bot the desired rating.

noobskull

I can beat 1700 bots when I am trying my hardest, I beat 1500 bots effortlessly

Kuso-ka

nope

Olbill
abigoda wrote:

I gotta tell you guys, i had a tough time beating 1300 Nelson...one thing i can tell is that at the assisted match this specific BOT made some ridiculous mistakes, and when i tried unassisted mode he was not fooling around. So my conclusion is that you can not use the assisted match mode to try to improve your game.

I agree. I normally play at assisted mode. If I play unassisted the bots seem to use entirely different openings and things get a lot tougher.

JasonJayMason

Last year I was holding ~850 in blitz, but when I got back into chess at the end of 2021 I have dropped to ~650 despite being a better player now. I now beat the 1300-1500 computers with a 3:00 timer almost every time. I think what has happened is chess has become more popular through streaming etc. so it is more competitive now-- and chess.com has not updated the bots to reflect the new ability levels of players. So in other words, the ability level of a 1000 player in 2018 it more like a 600 now due to a higher density of players who are playing competitively and are actively trying to raise their rank.

frangsbofarm
JasonJayMason wrote:

Last year I was holding ~850 in blitz, but when I got back into chess at the end of 2021 I have dropped to ~650 despite being a better player now. I now beat the 1300-1500 computers with a 3:00 timer almost every time. I think what has happened is chess has become more popular through streaming etc. so it is more competitive now-- and chess.com has not updated the bots to reflect the new ability levels of players. So in other words, the ability level of a 1000 player in 2018 it more like a 600 now due to a higher density of players who are playing competitively and are actively trying to raise their rank.

I agree with this and I also think the bots have somehow gotten worse.  A year ago Maria and Jade were giving me fits, and now I just beat Antonio and Wendy today without breaking a sweat, they just started throwing away pieces about 7 moves in.  This being with 5/5 time control.   And I don't think I've gotten that much better lol.  :-D   I'm 850 in 5/5 blitz against humans.

I went looking for forum posts to see if it was just me, and it's obviously not just me.

 

yusukeogawa

I can beat 1600 Isabel unassisted repeatedly and rarely lose, but 700 rated players beat me a lot. They are not accurate.

SRIegend

I am 750 rapid but I somehow beat all the  1700 bots.

asteener1867

I have beaten bots up to 1400, I haven't been on here for a while because I'm not, nor ever will be any good at Chess , but really enjoy the game.

I do seem to remember I got rated for beating them, either on this site or some other.

Its totally self defeating, because as soon as I tried playing people online, I had a rating of maybe 1000 and got totally slaughtered by most

pcwildman

I'm rated 1100 on daily and I noticed that I'm pretty much confusing the Isabel 1200 daily bot with simply E 4-D 4 openings, and then cleaning her clock. I don't think I've ever seen anyone retract a knight to its original position.

ninjaswat

Are bots' ratings accurate?

No, end of story.

tongue.png