It's a subjective thing putting ! or !! on a move in any game. If the anotator finds the move is brilliant or hard to find then he puts a ! on it. If the anotater considers the move to be extremely hard to find or exesively brilliant then a !! is in order.
? vs !
I don't put ! in any of my own games. I think that's the real criteria - if your opponent and/or third party observers think it was brilliant, not if you did.
Even with every day choices there are decisions and outcomes and their evaluations aren't dependent on each other. You can make an informed good decision and happen to have a bad outcome (walking to the health food store and get hit by a runaway bus), or you can make a poor decision and happen to have a good outcome (buying a lottery ticket and winning).
Moves that deserve a ! have nothing to do with what your opponent sees or doesn't see. They're usually hard to find/counter intuitive, and may be an only move. It's exactly what the position requires.
If your opponent falls for a trap he gets a ? but you don't get an ! 
I have only ever written ! once about my own moves and never against my opponents moves.
My opponents don't make good moves, they can make the best move possible but I don't consider it good. If they go on to win it's only because I missed something and so their good moves are a result of my own stupidity. Chess like everything else is cause and effect.
Understanding this is why I've only written it once against my moves. An endgame scenario where the key square had to be occupied by my king for me to go on to win the game, I saw it the opponent didn't. It wasn't a good move, it was the right move and I was just happy to find it because my opponent played well.
As for ? well my notation is riddled with it but I'm OK with that as it's usually after my opponents moves.
When annotating my games, it's easy enough for me to figure out when my move deserves a ? (or more likely a ??). But I am finding it difficult to know when a move deserves an !
I feel pretty much ecstatic every time I play a move that Fritz thinks is the best move, or at least in the top 3, but at my level of play finding the best move in a position is occasionally not hard to find.
Sometimes I may make a move that I think was brilliant, but only because my opponent overlooked a semi-easy tactic. So does I deserve a ! or does my opponent deserve a ? or both?
What does it take for you to dish out a ! or even a !! in one of your own games?