I usually don't like to win on technicalities, and I usually hate rules lawyers who come up with ways to win even though they have clearly lost. However, this doesn't seem like one of those situations. Your description doesn't quite match the diagram, but I think what he did was grab your pawn while touching his queen, or perhaps he picked up the queen, and touched the pawn with it?
Either way, he clearly expressed his desire to take your pawn, and moved the piece, then realized it was a blunder, and wanted to take it back. Bad luck. He was too late. He didn't see the mistake until after he was committed.
I sure wish I could take back a lot of similar moves I've made in tournament Chess, but I can't. In this case, I say there was absolutely nothing unsportsmanlike.
still seems a tiny bit unsportsmanlike...maybe I'm just too soft lol
I think it's the way some people approach the rules that make them seem unsportsmanlike, being rude to the opponent, and just waiting for the opponent to make a mistake and punish him or her for it because that's the only way they can win.
But in practice, touch-move is a very elegant way to play chess. It looks cool and avoids all sorts of ambiguities.
Plus, you have to draw the line somewhere. I was playing in a casual, class tournament in elementary school, and one of my opponents was so nervous when he was moving his Queen, he would sometimes be picking up my pieces (the Knight in particular) and moving it around and testing whether or not it could capture his Queen or not . . . I let it go, but still, this would cause all sorts of problems in regular, tournament play.
As others have pointed out, there would be all sorts of arguments regarding improperly placed pieces, "My Knight was on d5, your Rook was on c6!" "No, Your Knight was on d6 and my Rook is on c7!" [board being knocked over and pieces scattered] "Oh yeah guess what, now your Rook is on refrigerator 5!"
In casual games, I'm very soft too. And I hate to win off of a simple blunder. But in a tournament, everyone has to follow the rules.