Castling does 2 things:
1) Gets the King into the corner which means there are less directions to be attacked from and
2)Gets your R into the centre files where it can do the most damage, especially later when both Rs are joined.
However, I used to play regularily against a guy that would delay castling until the last possible second, because he was very strong at keeping his pieces mobile to deploy them quickly to whichever side I castled to. Eventually most of our games ended up without either side castling and my K would often move up to protect the centre pawns, but this is an exception for most of my games.
Take a typical position in the Caro-Kann that results in the King losing castling ability:
A friend asked me this. What do you think?