Komodo skill levels and Elo rating

I'm probably howling into the void here, but does anyone have a decent handle on what the levels on Komodo (as implemented here on chess.com) really mean?

Clearly the Elo numbers quoted are massively optimistic, by hundreds of points (at least around level 20ish). I'm wondering what they are based on.

This information is from Dragon, so is a bit higher than Komodo is, but should be in the ballpark.

• Dragon will drop about 125 elo for each point drop in Skill below 23 (24 to 23 drop is perhaps 200, 25 to 24 depends on time limit). Our intention is that for quick games (10 min, or 5 min + 5 sec increment) even a virtual beginner (say Elo 250) should be fairly matched with level 1, while the World Champion should have a tough time with level 23 (est. 3000 at 5’ + 5”). (Level + 1) * 125 would be a good estimate based on chess.com games. 

That may be what it says, but it just isn't plausible. It was playing really poor opening moves at level 20-21.

Check out move 3 in this game:

https://www.chess.com/analysis/game/computer/178378483?tab=explore&move=5

The only way a human over 1500 would ever play that move is a mouse slip.

The device is also going to have an impact on how well any given bot plays, as they run locally. I also don't think there's an opening book being used for the base engine, though I could be wrong.

I just drew against the top level Komodo bot on chess.com (and had a winning chance that I missed). I didn't even play very well. It's obviously been badly crippled in some way.

https://www.chess.com/game/computer/184407883

That's playing on chrome browser on a 2020 MacBook pro. Last night I beat it on level 24 - only one game out of a handful, but still, it's not something I would ever achieve against a proper engine playing at top level.

https://www.chess.com/game/computer/184050361

Jamesthetall, try playing against Komodo locally, in LucasChess app. When you start the game, you can set the Skill in advanced tab after you pick Komodo (it's in 3000+ category). There are 26 levels actually (0, 1, 2 etc up to 25 including 25).

I just beat the Komodo 21, lossing to it about 100 or so times and drawing to it 3 times. I have finally beaten the "Master 2500 bot".