bad puzzles, and reporting them

Not all of the puzzles are good puzzles. There are two kinds of puzzles which I find annoying:

1. This is winning because Stockfish says so. I had (not often) puzzles where the main line ended and the player was still down material, but the fish evaluated as +3 or so. Of course, the fish was right, but as a puzzle this is not very satisfactory, because that +3 eval still needs to be proven. The lines sometimes end early, apparently because there is more than one correct continuation, but it does not make a good puzzle.
2. More commonly, we have a puzzle that is objectively hard but becomes easy once you make the first move - because the engine response is poor. The engine sees all lines are losing anyway, and makes a rather odd choice for what the response should be. Example: https://www.chess.com/puzzles/problem/767426/practice Black threatens Qh4, with a mating net. The only winning move is Qb7, which you may as well guess. But then the engine response is Nc5 after which we simply trade queens, all fine, easy to find. But the critical line (from a human POV, not an engine POV) should really be what to do when black plays Qh4.

This particular puzzle leads to a second point of this post: Reporting bad puzzles. We don't have a way of reporting bad puzzles (some titled players seem to have though), all we can do is add a comment to the puzzle. This particular puzzle had critical comments going all the way back to August 2019. Is anybody ever checking these?

I should add that, yes, there is a way to report bad puzzles, I know.  But the reason for which we are supposed to report bad puzzles are limited to only incorrect and ambiguous puzzles. The two kinds I mentioned above do not fall under that. So, a more sophisticated report feature that includes a reason (and one could create a few standard reasons that covers most) would be appreciated.

If these reporting features are under risk of abuse by trolls, or you get too much WTF from bewildered and confused 600-rated players, you could safeguard against that by enabling such a feature only once players have done a certain amount of puzzles.

A variation of this kind of bad/unsatisfactory puzzle is when the line ends too early. Example: https://www.chess.com/puzzles/problem/514204/practice

The obvious first move is Nc6, and after Qc7 white has to do something about their knight. The incorrect (though also winning, +1.5) move is 2.Nb4. The correct move is 2.ed5 after which the puzzle ends, bewildering even some people who got this right. Why is ed5 so much better? Well, black cannot afford to take back on d5 (either way), because once the e-file opens Qe2 wins the house. However, that the player has seen Qe2 when playing ed5 is unclear, they may have retreated the knight a move or two later to b4.

To test this, the line should have continued with 2..ed5, an inferior move, but still the critical line for the tactic, to check whether white sees 3.Qe2.

I have caught several puzzles programmed in that punish you for using different solutions that lead to the same (e.g. check mate in 2) or sometimes even better results (e.g. retaining a queen instead of a rook). Wanted to report them but the desktop site isn't loading the only page that allows for that. What a mess.

Honza

While my pawn reaches the eight row I should have the option, which piece to select. Instead the computer ALWAYS arbitrary gives me only the bishop and subsequently takes my move as a wrong one, by which it makes bad it's own move not mine.

Thanks for creating this thread. I was about to do it myself. Some puzzles are very likely computer-generated and make no sense from a human point of view, especially when the cpu doesn't play the obvious critical response. I understand that chess.com might not have the manpower to review all the thousands of puzzles available on here but surely the ability to report the ones that could be improved on would go a long way.

Another example that combined the "line ends too early" and "computer move is poor" problem is https://www.chess.com/puzzles/problem/663750/practice

After 1.Re8 Nh4 2.Re3 the engine plays bewilderingly 2..Nf5, relinquishing the piece with no resistance. Why? After the more natural back capture 2..fe3 the knight is trapped, and can be picked up with a king walk. Did the player playing the first move (2nd is forced anyway) see this? Unclear. Can we rescue the puzzle by playing the critical response instead? Not really, both Be4 and Kf6 will do as a continuation, and although Be4 is slightly better the difference is marginal and players may lose a lot of time agonising about the difference when both are winning.

Another example of a "bad" computer continuation: https://www.chess.com/puzzles/problem/978580

After 1.e7 a2 2.e8Q the engine decides to trade queens with 2..Qd7 which from a human POV is obviously hopeless. It just happens to be the right thing to do when you play bullet without increment against stockfish. The critical line is 2..a1Q after which white has a forced mate beginning with Qg6. Even if I saw the threat as the defender I would still play 2..a1Q against any human player.

The tactics don't have to chose what you think the critical line is and it is very possible that another tactic is in the system where your suggested critical line exists. I have seen a number of tactics from various lines in the same base position but a different counter is attempted.

What one player is worried about in a particular line may not be what another player is worried about and neither one might be the objectively best counter. All that matters is finding the best reply to the opponent's chosen line.

Also, some tactics end where they do if the continuation may have multiple good enough lines, since the tactic system was built with only one possible solution.

The puzzle page for any particular puzzle has a report button

@elefantenvogel Honestly, the puzzles you provide actually have a reasonably identifiable justification for the series of moves by the computer. I won't go through each one, but as an example, in your first post, the puzzle in the second bullet, you say that the 'critical' line is how to respond to Qh4. This is correct and the way to address the threat of Qh4 is to prevent it from happening in the first place via forcing moves. After trading queens, you have a winning advantage. As for the puzzles you say that they end too soon going into analysis mode allows you to see the continuation if needed. Unless there is clearly only one reasonable response, then continuing puzzles beyond the initial forcing moves would cause a reason to report (as in being too ambiguous).

OK I haven't read the posts before this, as I assume that it will be too high level for me who has only been playing chess for a small time.

But I wanted to discuss this puzzle, this is the opening screen of the puzzle & I think it is too naughty:

I will tell you why:

 Because even though I don't know much about chess, I can tell what the first move would be, as every other move LEAVES TOO MANY OPTIONS for black.

Simple as that.

I think the puzzles should not be like this, it should be not this thinking "well chess.com is going to want the puzzle to have only one most logical move, so anything that creates 2 moves for black is going to be wrong, so whatever I do must only have one (kind of forced) move for black." That means I must trade queens here.

Does anyone else see what I am trying to say? My logic on the number of options for the opponent shouldn't get to take precedence over my chess knowledge when doing puzzles.

I shouldn't be able to make the correct first move based on this logic, when I don't have any idea what the result of the puzzle is going to be.

Yeah but in most cases, most human players can reasonably agree on what the critical line is. Every time I wonder what would happen if a particular series of moves were played after the puzzle ends, I scroll down to the comments and see that one or several people have already asked themselves the same question. Unlike what your post implies, it's not a subjective thing; similarly rated players will think of similar lines, it's only logical.

Moreover it should go without saying it's pointless finding the best move if you don't understand why that move needs to be played as you could find it by chance.

@wollyhood I don't know that puzzle's rating but, assuming it's a low-rated one, I believe the point is merely to notice that your queen is hanging on h5, which is something a low-rated player might overlook.

@wollyhood Puzzles (tactics) are generally designed to be a series of moves which bring a player some sort of advantage. It could be checkmate, winning material, not giving up checkmate, etc. Yes, the first move, as you said, often seems clear because an obvious advantage is gained after one move or possibly two. This is most common on lower level puzzles. However, as you progress in your understanding and success with lower level puzzles, you will find that as puzzles advance they will start to require 3-4 moves to gain an advantage. This requires you to be able to 'see' the how the board looks after the next move for each you and your opponent and repeat that 3-4 times, rather than just 1-2 times. Each time you will have to determine what the best move is for your opponent. As I got better with puzzles (and still have a long way to go), I found that just understanding the first move was only half the battle. Being able to see the next move and beyond before I made that first move was the key to getting consistently better with these. Using the analysis function on puzzles I didn't get right or didn't fully understand made me realize I was not thinking far enough down the path.

Maybe. I know when I've discussed puzzles within my in person club in the past, we often have different thoughts on what the critical reply was or what defense we were most worried about.

So, it isn't as cut and dry about it, so here, what matters is finding the first move in the line and hopefully you have calculated all valid lines to quiescence to make sure your move is valid. I certainly don't always do that and will analyze if there was a reason why the line I was worried about wasn't played, to figure out if I was missing a resource or something.

oh. there were 3 or 4 moves involved though

fairy nuff, point taken : )

Yes, but the winning advantage after trading queens is easy to see, and that qh4 was prevented by a forcing move is not. Thus that should be the critical line, ie. play Qh4 regardless and test the forcing moves. Note that my complaints here are from the pov of puzzle rush. Here it is annoying when you agonize over a tricky puzzle, play eventually and cluelessly one of the moves that offers some hope, picked the right one, and then the engine sees too far and plays something that offers no resistance, because at the end everything is bad.

Another example: https://www.chess.com/puzzles/problem/999832/practice This is apparently a supposedly easy puzzle, as I got it as 8th puzzle in a rush. I didn't see the win, but eventually played the correct Qe7 as the only thing worth trying. The engine played the no-resistance move Kb8, and after Qxd8 it's over. Doh! What I was agonizing about was what to play against Kc8 - there is no knight fork because of the pin. The solution is simply to unpin with a quiet move: Kc1, and black is helpless. But that was not easy to see, and is moreover an unexpected nuance for a low-level puzzle.

@elefantenvogel I understand what you are saying and feel frustrated at "low-level" puzzles in puzzle rush too. However, I think you are looking at it from the wrong perspective. I often will go back to a puzzle via the engine and figure out why the move I thought was supposed to happen, or was supposed to be critical, wasn't any good. Doing this is a reflection of my playing strength (not very good). When I watch people rated 2000+ on Twitch or whatever I can't recall any of them running into the same struggles with this and they are able to address these problems immediately (Qh4 is not a threat). This tells me that their expertise has already taught them that defending against Qh4 is unnecessary. My inability to know or see that, especially in a puzzle rush scenario, tells me that I need to keep studying and practicing rather than looking at the puzzle as as too tricky for what it is rated.

Looking at the new puzzle you provided, it is truly over no matter what the computer does (Kb8 or Kc8) as the black Queen is pretty well stuck where she is. I think with experience you, and I, will be able to see those nuances faster. Personally, I gave up doing puzzle rush too much because I realized I didn't know basic puzzles well enough to get anything rated higher than 1000 (though my puzzle rating is 1600). It is one thing to be able find a solution at my own pace and it is another to do it in blitz/bullet time crunch.

Well, my puzzle rating is 2122. Qh4 in that puzzle is a threat, it needs to be dealt with, it is just not clear that the main move does deal with it. Note: I did get that puzzle right, but I did get it right for the wrong reason, and that wrong reason (the critical line IMHO) was not tested. And it does happen to stronger players, it is just an issue when. Have a look at the 993-rated puzzle IM John Bartholomew failed: https://www.youtube.com/watch?v=YMUqHZbI-0c (he re-visits at the problem at 10:50 in the video). John did not find the winning move Qa8, because he didn't see what to do after Be8. The puzzle is low rated though, because the engine responded with the low-resistance Rc8 instead, and then you just take. After Be8 white had to find the subtle Qe4, and that was definitely the critical line. So, many people would have got that puzzle right for the wrong reason whilst John got it wrong for the right reason, because he did not see the followup. When you do puzzle rush or play rated puzzles that sort of thing is very annoying.