Open the center and attack that king!
Exploiting Opening Errors: Not Castling

It depends on what opening he's playing. There are openings where it's fine not to castle for a long time because your position is too solid.

I can think of a couple opponents I know (very strong players) who deliberately avoid castling in lines where it would seem like castling is very important, namely the Sicilian Dragon! The key is to see how it deviates from mainline... in the mainline Dragon, white's plan usually revolves around an h4-5 push, breaking open the file and attacking the king (Sac sac mate, as Fischer put it). If black has not castled you must realize a couple things (this applying to most openings)
1- A kingside attack would look rediculous.
2- With the king in the center black will be unable to prevent or 'punish' pawn breaks that are normally impossible in the Dragon ...
therefore white's plan must change (while still being flexible, because black can castle when he wants)... A g4 push, preparing g5 (hitting the Knight) will allow white to gain central control with Nd5 ... That in itself is the 'punishment' for not castling. If and when black finally does castle (he'll need to eventually) the g4 move will still be useful!
So in short:
If possible, break open the center! So long as your opponent hasn't castled, tactics on the king will rule the day in an open (or openable) position.
If the pawn structure is locked (This is common in the French Defense), begin operations on 'your side' of the board and don't worry too much about the king... he'll be compromised eventually.

ih8sens is right. Attacking a central king in the dragon is quite confusing at first, in fact one time in a schevinegen (but my opponent castled queenside) I kept my king in the center and he didn't know what to do.

Thanks to all of you for the great advice! Esspecially you, ih8sens, for the detailed analysis. :-)
Ironically, I just looked at one of my games (currently at move 14) where I'm in the middle of punishing an un-castled king. :-) You can see the game here: http://www.chess.com/echess/game.html?id=24982394#

With the existance and playability of the boungcloud, I find it difficult to convince myself that not castling is necessarily an error.

Looking at your post about the bongcloud "attack" I'm not sure I'd take a page out of Lenny_Bongcloud's book, seeing as he IS famous for having the lowest rating on the site.
I sometimes forgo castling, especially when playing weaker players, in order to maintain the tempo of a strong attack. But generally when playing equal or stronger players it seems harder to protect the uncastled king.
Hey, I'm about to start a team match against a guy who I noticed (looking at his games) usually fails to castle in his opening. How do I exploit this weakness? Any tips/strategies you can give me?