No....DON'T be annoyed. SAY it.....say "Great game". Give your opponent the credit he deserves for the win. Be a good sport, a gentleman, and a good human. Don't require or expect him to respond in kind. What you do is good and the world needs more of it....
For every one who doesn't respond, there will be many who are touched by your gesture and will maybe show that respect the next time they get beat.
If someone congratulates me for beating them, I enjoy the congratulations more than the victory, because it's a genuine human connection. Don't underestimate it.
I hate getting beat, and you say that was an awesome game, but it's like your opponent is stone cold dead, I mean jeez, great game, but like wha? I thought we were supposed to be friendly and sharing tips and tricks? So annoying...