Blogs
State of Chess AI – ChatGPT vs Gemini

State of Chess AI – ChatGPT vs Gemini

Avatar of sbernst8
sbernst8
| 0

State of Chess AI – ChatGPT vs Gemini

I continue to be curious about the state of AI and chess.  As far as I know, only one site (DecodeChess) offers an AI to analyze your games.  I was curious how they do it so I went exploring.  First, I used my company's site Knightly Chess and found my worst move in the last 3 games I played. Next went to OpenAI's ChatGPT and Google's Gemini and input my FEN and move to see what they came up with. 

Prompt:

FEN of Position

The board looks like this:

Board Setup

With white’s bad move of Bg3 instead of taking either exd5 or exf5 my evaluation drops from +1.6 to -1.7 quickly.  But let’s what ChatGPT and Gemini say about the position and move

ChatGPT:

Move Analysis

The move Bg3 means moving the bishop from f4 to g3. Here are some considerations:

  1. Purpose and Objectives:
    • Bg3 is a retreat of the bishop, aiming to preserve the bishop pair.
    • It might prepare to push e4-e5 in the future, undermining Black's center and knight on d5.
    • It avoids any potential threat or exchange on f4 by Black.
  2. Tactical Considerations:
    • The move does not immediately blunder any material.
    • It maintains the bishop pair, which can be advantageous in open positions.
  3. Positional Considerations:
    • The bishop on g3 is well-placed to control important squares, like e5 and h4.
    • It supports potential pawn breaks on the kingside.
    • It may help in defending the king if castled kingside.
  4. Opponent's Potential Responses:
    • Black might consider moves like ...Bxe4, but that would likely be met by Nxe4, keeping the position balanced.
    • Other plans for Black could involve ...Nf6 to challenge the center or prepare for castling.

Conclusion

Bg3 is generally a good move. It maintains the integrity of White's position, keeps the bishop pair, and prepares for potential future breaks in the center. It does not create any immediate weaknesses and maintains flexibility in White's position.

So, Bg3 is a good move.

 

Gemini:

I can't definitively tell you if f4g3 is a good or bad move just by the FEN and the specific move itself. Here's why:

  • Limited Context: FEN captures the board position but not the history of the game. Factors like previous moves,castling rights, and en passant opportunities can all influence the evaluation of a move.
  • Calculation Required: Assessing a move often involves calculating several variations (possible future moves) to see their consequences. This is complex and computationally expensive, even for strong chess engines.
  • Strategic Understanding: Good chess moves often involve strategic concepts like controlling the center,developing pieces, and king safety. Evaluating these requires a deeper understanding of the game beyond just the immediate consequences of a move.

However, I can offer some resources that might help you decide:

  • Chess Engine Analysis: You can use a chess engine like Stockfish or Komodo to analyze the position after f4g3.These engines will suggest the best move and show alternative lines with their evaluation. [Consider Lichess analysis board - lichess.org]
  • Online Chess Resources: Websites like Chess.com and Lichess.org offer tactics trainers and endgame studies that can help you develop your chess vision and understand how pieces interact.

If you're interested in learning more about the specific position you provided, I can share some general observations that might be helpful, but keep in mind these are not definitive:

  • The white pawn on f4 is currently unprotected. Moving it to g3 might weaken its position.
  • Is there a specific tactic you're trying to achieve with f4g3?

By analyzing the position further and considering these points, you can make a more informed decision about the quality of the move.

 

Reviewing the output

ChatGPT Review

All I can say is wow.  Not even close to a correct answer but ChatGPT seems so confident in the response. 

Gemini Review

It is interesting that Gemini has the knowledge to say I should use a chess engine like Stockfish or Komodo (the two most powerful engines) to analyze the position and also offers up some online chess resources.  But also, it is completely incorrect.  First, provided a FEN, in order to understand if a move is good or bad, I don’t think you need the previous move history.  Simply looking at the board and evaluating the position and the move is enough to determine the validity of a chess move.  Second, it says there is a white pawn on f4 (which there is not). 

Final Thought

The state of Chess AI is still far from useful.