I only ever get offended when my opponent says GG after the first move or two, other than that I just assume it's a platitude for "Thanks for the game". Just like "GL" doesn't actually mean the opponent is wishing you luck because I assume they don't really care enough.
Of course, this thread and many others have shown that there are people who are thrown into an absolute frenzy over "thanks for the game" and "gl" too and will provide all sorts of reasons why those phrases can be interpreted as rude.
It's all just people looking for excuses to be offended. Let them wallow in it-- they're happier that way.
What's 'gg' mean when you use it after a loss?