It's a rule of thumb only. It would be more correct to say that:
1. Normally, on each side the knight should be developed before the bishop.
2. Normally, you begin with the side you plan to castle to.
3. Heed the exceptions!
The reasoning behind the rule is that, normally, the good places for your knights are more limited than the possible good places for the bishops. So it is a good idea to wait a bit before committing the bishops and decide it in context of what your opponent does and what your general plan will be. Actually, that's exactly the same reasoning as of why you wait even longer with developing your queen! The exceptions would be tactical reasons or opening lines where it is vital that your bishop goes somewhere specific - like, say, certain fianchetto openings.
"I have added these principles to the law: get the Knights into action before both bishops are developed." - Emanuel Lasker.
Thoughts?