The touch move rule is there simply to make you a better player. You won't make (as many) impetuous moves because of it.
I Don't See the Point of the Touch-Move Rule

I do often adjust my pieces at the start of the game, especially if I didn't set them up. You say "j'adoube", or "I adjust" once, get on with it, then make your move. No big deal, no big production, no silliness. Just play the game.

It makes no possible, logical difference whether you touch the rook or the king first when you castle. If Fide requires the king to be moved first, that is illogical. On what is it based?
It's based on castling being a King's move. If you make a mistake and fail to notice that castling would be moving through check (for example), you would still be forced to move the king, unless you were playing by USCF rules and touched the rook first.
If you look at the definition of the move, castling is the only time the King can move two squares. It's an artefact from a version of the game that allowed the king to flee two squares once a game, and it modernized the game by allowing the king to flee the center once the B and Q became the powerful pieces they are today.
The rule is logical.

For those who have no clue what SilentKnighte is referring to, Kasparov, in a blitz game against Judit Polgar, once let go of a piece on a square that blundered the game. He instantly grabbed it and made a better move with it. The incident was caught on camera and is available on youtube. You have to watch it in slow motion to see that he definitely let go of the piece. He ended up winning the game. I don't think Polgar ever forgave him.
And, apparently, neither have some people on this site.
Magnus did the same against the Chess Queen.
http://www.youtube.com/watch?v=WeyXKTVYenA
around 6:29.
"Chess queen"
Well, have to give her credit. She managed to flaunt being female in something that's completely gender neutral.

That's because e1-g1 is the shortest notation that implies castling. Also, you touch the king first for touch move purposes. As SmyslovFan said, the illegality of castling is based on the king being in check. Also, if castling is illegal, you should be bound to move the king, as Rh1-f1 wasn't illegal, it was the king's journey to g1 that was against the rules.

I always thought you tuched the king first to signal there is more to the move? If you move the king two squars its casteling, if you move Rf8 its a ordenary move?

MrHjornevik, you were taught the "right" way.
Whether you play USCF or FIDE rules, it's better to move the King first. Spain follows FIDE rules anyway, so in a tournament in Spain, you could be penalized for moving the Rook first. Again, I doubt any IA would actually punish an amateur the first time he did it in a tournament anyway.

You have to touch the King first if you have to castle. As touching the rook first have the possibility to move beside king's square.

I used to touch many pieces at the same time before I move time and say to my opponent that I will only adjust it. I see him distracted and a little bit annoyed. I don't get disqualified either.

It makes no possible, logical difference whether you touch the rook or the king first when you castle. If Fide requires the king to be moved first, that is illogical. On what is it based?
It's based on castling being a King's move. If you make a mistake and fail to notice that castling would be moving through check (for example), you would still be forced to move the king, unless you were playing by USCF rules and touched the rook first.
If you look at the definition of the move, castling is the only time the King can move two squares. It's an artefact from a version of the game that allowed the king to flee two squares once a game, and it modernized the game by allowing the king to flee the center once the B and Q became the powerful pieces they are today.
The rule is logical.
Moving the rook first is also a complete move as soon as you remove your hand from the piece. Moving the king first requires that you move the king more than one square, something that's only possible in when castling. Requiring that the king be moved first when castling is the only way to ensure that there is no ambiguity in the context of "touch move".
The comment above about your opponent's reaction is also spot on when it comes to the touch move rule itself.
What annoys me is a player who moves his piece to a chosen square and keeps his finger on it as he surveys the board. Then he moves the piece back to it's original square if he changes his mind. Evidently the move is not completed until the piece is released.

What annoys me is a player who moves his piece to a chosen square and keeps his finger on it as he surveys the board. Then he moves the piece back to it's original square if he changes his mind. Evidently the move is not completed until the piece is released.
my brother does this. I let him get away with it more often than not, I also let him take back moves if he scream "no" quick enough :) Once he start beating me I might change, but for now it works ok

What annoys me is a player who moves his piece to a chosen square and keeps his finger on it as he surveys the board. Then he moves the piece back to it's original square if he changes his mind. Evidently the move is not completed until the piece is released.
They should make another rule which would forbid you from taking your move back when you intentionally land your piece on a square, regardless of letting it go or not.
Why are fire engines RED??
Is this one of them there Mensa test questions? (Never mind that they're not all red; OK, most are so I'll give you that one.)
The fire engine makers always have a surplus of red paint?
They're really supposed to be white, but someone keeps spilling huge amounts of Heinz ketchup into the paint vats?
Did I pass? Please tell me I'm not one of them there dumb ones...
http://en.wikipedia.org/wiki/Fire_engine_red
I still think it's the Heinz ketchup theory. What does that stinkin' Wikipedia know?