Talking of AI, I have had some interesting experiences related to chess. I asked AIs to find sets of good exemplar games including PGNs in chosen opening variations . At first I believed them (typical example would be referring to the 1992 Fischer Spassky match game 3 and then giving the full score of the game),
As well as some red flags in the quality of play, in one of them I found a plain illegal PGN. One player left their king in check.
This is when I found that there was a lot of hallucination going on. The latest AIs are quite good (but not perfect) at generating plausible looking (but not very high quality) games, with occasional plain illegality, but they are often not real games at all, despite their descriptions. They even lie about games from matches as prominent as that Fischer-Spassky one.
Oh well...
Of course. I said only a human.