[Public Preview] Frostbite: The spellchess.win Engine

I've been slowly working on a Spell Chess engine since last June. While it's certainly not done, I have made some good progress and the current version could be useful. It certainly sets the table stakes for engine strength. With the recent spike of interest in engines, I'd like to share this preview, in-development version of Frostbite. But first:

Fair Play

Obviously, you may not use the engine (or the site at all) during a game. All requests are logged as inputs to an automated pipeline to detect engine usage during games. Some information is provided to chess.com, but for fair play security reasons I won't say exactly how this works. Multiple accounts have already been closed in connection with the site, including for cheating with a private beta of this engine. Do not cheat; you will be caught.

Play Frostbite

This special beta link will allow you to see the Engine tab: https://spellchess.win/?beta=0abf9a6d0bf1

To play against the engine, set "play engine move" to "for white" or "for black." If you just want to analyze a position, press Evaluate Now or check "auto evaluate" (the equivalent of turning on an eval bar).

As this is a preview, I might end up disabling the link temporarily or longer-term if there are problems with fair play or server overload.

Strength

Frostbite does not yet have enough depth to provide broad opening insights. In fact its positional play is sometimes strange. The regular version is stronger than NNUE and less likely to fall into opening traps, but NNUE may be more fun and human-like.

But overall, it would be a decently strong player: I think over 2000 if given 5 seconds to think per move. If I play it while distracted doing something else, it crushes me. Even when I'm paying attention, I often lose to it from common openings. A human+computer team with a 2100~ human to guide the engine would very likely be 2500+. Even a weaker version from last August was helpful to me for opening prep (never during games, of course).

I hope it's useful to you as a sparring partner and second opinion in analysis.

This is version 0.2. My plan is to set the version to 1.0 once it's superhuman, able to consistently beat the strongest players. I hope to make more progress and see others' engines in the coming months.

Congrats on the release and thank you so much for your contribution! I think it will be super helpful for the enthusiasts in the community that want to do either some opening prep or just some deep analysis on a game (for this end, I guess it'd be super good to be able to paste the move list from chess.com to set up the position instead of manually replaying the game, but I can see how that would get exploited to cheat). Happy to set the bar for "superhuman" if you ever want to test a stronger version of it lol.

I'll run some tests, but I have a question already: What does NNUE vs non-NNUE mean? Also, when you say "with book", does it mean that it will follow the most played moves in the database for the first few moves?

Nice!

Wow, congrats Fleex, this is gonna change a lot of things in spell chess, for exemple i already see that he considers e4 e5 freeze@c7 Bb5 like way better for black which is mind blowing (I bet he’s not gonna love your way of using spellls every where, TwoPotionsOneMate :)). Thanks a lot, and I hope we will be able to collectivly study the spell chess tactics and opening with this engine’s help.

It's very strange. If after freeze@c7 Bb5 in the line Delhy suggested black plays Ke7 the computer will think that is much better for black but if you let the computer play against itself, white will mate within 6 moves, at least the three times I tested it out.

No way we already made the engine bug, we are demolition men.

Indeed, I would be skeptical of its opening evaluations at this point. It will definitely miss long-ish attacking ideas due to depth.

@TwoPotionsOneMate You can paste moves, but only in the PGN4 format available after the game. Switch from the Chat to PGN4 tab on chess.com and use the Load PGN4 button on spellchess.win.

The NNUE version includes a small neural network in its evaluation function. (This is a tiny NN compared to Leela/AlphaZero, and not as big as Stockfish NNUE either.) This makes the engine more human-like in some ways, but causes it to miss different things. The non-NNUE version has very limited positional understanding, which can cause some bizarre moves.

"With book" means the engine can consult the opening database to pick a move in common positions instead of doing calculations. It picks randomly between common good moves to add some variety. You can force the engine to calculate with evaluation mode. Though again, opening calculations will probably not be very good right now.

Personally I've found it more useful to practice watching out for tactics, where I often blunder. I'd be interested to hear how its playstyle works against others.

@Fleex255 However you said you used your engine for openening prep so it has some opening understanding.

Some, yes - certainly not enough to trust without experimenting. It was especially good for finding/confirming traps like this and this.

This is awesome!

After playing around with it for an hour, I think 2000+ is a pretty conservative rating, I would personally give it closer to 2300, since it outplayed me pretty consistently in most positions.

But it sometimes fails to see tactics, that take 3 moves to execute (for example the freeze Qxf7, Bh5, Bx King tactic).

Can you maybe set the depth to at least 5 for move sequences, that start with check, or put pieces on sqares, from which they can (jump) capture the king?

I would guess, that you prioritize forcing moves already, but then, I know nothing about coding.

Anyhow, awesome work!

I lost my first game against NNUE (5s/move):

My first loss! (engine as black): #!moves=DBwyKhwkNCwFDD40/Qwntz8nAyfeNi6IADAuJzA4OSjzARLwPTYGF7o7OAsbKBkSHMUZCtIcKzw9BAMKGxcmNiQmNz0+7DctPjYtJzY+BwY0Lis6ODoNHSQ2JzY+NgkRLCQdJDo/Agk/DwkbNj4bIjEhAwo+NwoBLiQiGyQuCBgqIhsiIRgRGDUlGCAPOSIpOTgBCS4kBgc3PgcGPjUJAMUkFQAJFQYpBjgOCRIOBg

Okay, I went way too aggressively on my opening and blundered hard, but it's still VERY impressive that it's already that strong.

Nevertheless, I manage to easily win the games most of the time because the engine fails to see a lot of tactics on the opening (I guess due to the depth). Examples:

Freeze+jump madness (engine as white): #!moves=DBw0JAUaPSL1GjXtOy2IADAtNQYXODAIEDUuAww5KhcdyyQdCRk+NBkiKhsMA8wbCsEiKi4OAwoOBwQMNCoKIh0VDBPbBwXMAgkzKxMUKyK2CT/UBQLDARIwED82khAU

Ke7?? (engine as black): #!moves=DBwyKhwkNCwFDD4v9gwnPDQGFzsyCxszIwESNDMEBi8l6xIj2z0rjQUlKiMkKzIpJTUzKwIdKyrxNTLpLCQnPCoypB0y

I'd say if you manage to improve the openings, it'll be the perfect sparring partner to prepare for the world championship. I can easily see now how this engine will beat me every single time as you get closer to version 1.

Awesome work, looking forward to seeing further improvements and helping as much as I can : )

Edit: I can't add hyperlinks for some reason, so I'm pasting the full URLs

5s/move is really strong i almost drew it once but I threw…

Well, my sample size is small (I played like 4-5 games), but my feeling is that it's still so flawed in the opening that unless you blunder hard (as I did in the first game), then it's hard to lose a game against it. But besides that, it already looks 2400ish

The rise of Spell chess engines!! Any pointers on how to access it for analysis? Is there any website? Can't find anything on the Stockfish one

I'm a bit rusty, but I was able to beat NNUE with 5 seconds the first time it played 1...e6 Do you think the mistakes it made were due to bad opening book or due to blunders the engine itself made?

After analyzing, it seems the engine thought black was better or at least equal until after I play e5, maybe it undervalues initiative a bit.

Also not sure if bug is the right term to use here, but there are some edge cases where the rules of spell chess don't align with what the rules are here on chess.com. For example in this position, J@e7&Qxe8# is not a legal move on chess.com, but that's more of a bug on chess.com's part (hopefully they'll fix it at some point).

Thanks for playing! Yes, it definitely fails to understand most positional concerns (in the opening at least) and initiative. e6/Ke7 is playable in my experience, but it takes positional understanding to survive and keep the advantage.

The pinned piece with check edge case is a bug in chess.com IMO, so I expect they will fix it... eventually, hopefully. It's from a mistake in their move generator, and since my move generator works differently, it's not possible to replicate chess.com's behavior without some extra calculations. The performance cost would make the engine measurably weaker just to add a bug that almost never comes up.

Huh, I was not aware of this. I did check before I started developing, but I see now that there is a pending pull request to add support. (I still was not able to find any official support, though it's possible I missed something.)

I'll have to try it out to see how strong it is. Hopefully I'll have time later this week. If it's very good, my job will be a lot easier since I'd just have to connect it to my GUI

Fleex already told us his app wasn’t doing the sae bug as in chess.com yeah