Are Bot's ratings accurate?

Sort:
R0swill
Anonymous_Dragon wrote:

From what I have been reading about everyone's experience here and on other threads , I think its safe to conclude that the Beth Harmon bots are overrated. I mean all bots at lower levels are in general....but the Beth Hamon bot just goes a bit further and is easier compared to the other bots at her level . And I have seen people 500 rating points below hers defeat her. So am quite optimistic and will try to take on the 2000 rated Beth .

Its not a mistake they know a lot of new people are joining chess and they are making these bots easier than they should be.

NikkiLikeChikki
So players at different ratings on average have a different mix of best, excellent, good, inaccurate, mistake, and blunder moves. The bots try to duplicate this average of what actual players do. So in a way, they are accurate, but I don’t think the bots are very good at determining in WHAT positions players of different ratings will have trouble. Sometimes they will make a dumb move in an obvious position and a best move in a complex one. But they are a lot better at seeming human than they used to a few years ago.
Vibhansh_Alok

Ok so...Beth (2400) is some real challenge for me...below that it’s easy if you’re intermediate....about the 800 rated beating much higher rated bots...it happens.... sometimes bug (you’ll know by looking at analysis) or maybe random play time...few weeks ago a rumour got viral that Magnus age 29 (chess bot available exclusively on Play Magnus App) resigns to even new players by making some seriously unexpected blunders...I didn’t pay much attention over it that time but soon it got a covid viral...a lot of guys started to claim victory over Magnus (29)....I’m not sure if they’ll are lying or there is some serious bug.

Vladimir_Sirin

It lends credence to the old saying: "To err is human, to really foul things up requires a computer."

NikkiLikeChikki
Beth should definitely resign if the computer says she is down -10. Mr. Shaibel would be very unhappy with her if she did not.

Hear that folks at chess.com?
kungfiske
Hack89 wrote:

Isn't it sort of fruitless to compare blitz ratings against bots when there's no time limit? Maybe it doesn't make much of a difference when you're no longer a beginner, but I have a very low rating of 600 in blitz and seem to play on equal footing with bots rated 1300 (played it a few times but it's the highest rating I've tried so far). Seems to me like playing with or without time constraints are two very different things that can't really be compared.

This is my experience as well. I'm just getting back to playing chess and my Blitz rating is abysmal but when I'm training against bots, I seem to hold my own or remain competitive against bots rated 1200 to 1600. The time pressure in Blitz makes all the difference. 

neveraskmeforadraw

Bots ratings are slightly higher than otb fide bot ratings, but only slightly so you should be fine.

tatuzudo

I'm around 1300 here and was able to beat beth harmon 1880. I think there's something wrong with those bots. Didn't played the other bots to compare though.

Vibhansh_Alok
NikkiLikeChikki wrote:
Beth should definitely resign if the computer says she is down -10. Mr. Shaibel would be very unhappy with her if she did not.

Hear that folks at chess.com?

Character in The Queen’s gambit, Beth’s owes a lot to him...

Vibhansh_Alok
tatuzudo wrote:

I'm around 1300 here and was able to beat beth harmon 1880. I think there's something wrong with those bots. Didn't played the other bots to compare though.

But if you see bots at higher level like a bot (3100), Naka can’t beat this level bot despite having rating above 3100 at chess.com.

Dinch-Owl

Hypothesis: The bot's difficulty is affected by the skill level of the human player.

Evidence: I've seen a number of highly skilled chess streamers such as Anna Rudolph find difficulty defeating the 1880 Beth bot; yet I've found the 1880 Beth bot to be very easy for me, even though I am a patzer.

How could the bot be affected by the player's skill level?

One possibility is that the bot retrieves info from your account and modifies difficulty on this basis. For example, it might look at your bullet/blitz/rapid ratings. If it's playing someone with a high bullet/blitz/rapid rating, it plays better than it would otherwise.

Another possibility is that the bot simply looks at the quality of your moves during the game you're playing with it. If you make good moves, the bot makes good moves in response; if you make blunders the bot makes blunders.

Vladimir_Sirin
Dinch-Owl wrote:

Hypothesis: The bot's difficulty is affected by the skill level of the human player.

... the bot simply looks at the quality of your moves during the game you're playing with it. If you make good moves, the bot makes good moves in response; if you make blunders the bot makes blunders.

I believe this is exactly what is happening.

NikkiLikeChikki
One of the complaints about computers is that they don’t play like humans. While their moves are good, they are often weirdly counterintuitive. Especially at lower levels, training against a computer doesn’t really prepare you for chess against humans because the engines just play completely differently.

AI chess engines typically are more human-like than traditional engines, but are still oddly counterintuitive sometimes because they learn from self-play. Maia is a new AI that is trained on actual human games—millions of them. The 1100 elo AI is trained on games played by 1100-ish players. There are also a 1500 and 1900 versions. The data shows that Maia more accurately reflects human moves.

Pretty cool, right?
Michael_Ska

Same, I beat Hamon on ratings way higher than mine... She pretty often do INSANE blunders...

Anonymous_Dragon
JosiahRichardson wrote:
Do bots give u rating?

Nope they dont.

Juan-soIo

Ratings are slightly skewed on bots due to the time not being there. My rapid rating is 1382 yet am able to take down 2000 rated bots. This is because I am far stronger under no time pressure. Having the freedom to explore all the variations i would like gives me the ability to beat the higher rated bots. The actual ratings of the bots are fairly accurate however, just because you beat a bot far higher than your rating one time, does not mean you are that good; the bot could have blundered a winning position or various other things. You should be able to beat any bot within a few hundred rating points of your rapid rating. If you find yourself consistently beating far higher level bots, try and play similarly rated bots on various different platforms or get premium here and play the vast array of bots chess.com offers of similar rating.

Vibhansh_Alok

But high rated engines well represent their ratings, like stockfish, Lc0 etc...

Anonymous_Dragon
Vibhansh_Alok wrote:

But high rated engines well represent their ratings, like stockfish, Lc0 etc...

yeah... The simplest way I can think of it is that lower rated bots are just the same higher rated engines that are asked to through in some mistakes and blunders occasionally in the game based on the rating .

landrew_JR

I love playing bots. My dad does the same

 It allows you to get back in practice if you have been out for awhile. My dad loves playing anna Rudolf. He rarely wins but has fun. We both find the bots fairly accurate. Especially in advanced and master levels. 

TheGChess
Yea I played a chess bot at my level and I felt like I was playing someone who had never played before