I think the disadvantage is a small one.Probably has do something with weakening of the light squares in whites position.Also i noticed that white is allowing black to get rid of his bad bishop for whites good one.
4.Bd3 in Advanced Karo-Cann? (What is wrong with it)

You must be new to the CK. This is a very common move amongst weak players, I would say that it is the move that I face the second most frequently from said position. It makes some sense to exchange off this Bishop that Black has placed outside of his structure, at least that is what I imagine goes through the minds of the people that play this move.
There is a simple counter, but it doesn't win any material. It just leaves Black with a nice position.
White has a few minor variations on this line but only one is an improvment. He can play 7.c3 or Nd2 for similar (probably slightly better) positions. He can play 5.cxd3 or 8.0-0 but both lead to doubled pawns that he will not be able to get rid of. He can also chose to simply move his Queen (misplace it) to avoid the exchange but he will struggle to castle for many moves to come. Note that he can't play 8.Qd2 because of Bb4.
The position in the above diagram is better for Black because he has the better pawn structure for an endgame and all of White's attacking chances died with the exchange of Queens. You will be playing moves like Nc7, Rc8, c5, Ne7 from which it can jump to it's usual g6 or f5 squares or the usually even better c6 square. You simply have a very nice game.
The only real way for White to claim that his opening wasn't a complete failure after 4.Bd3 is the following line, which you may like to note, I have never faced:
That diagram shows the line that Black should play to keep approximate equality. If you try the same Qa5 line you won't achieve your deseried results because of the placement of the White Knight which now both covers the King for castling and opens up more squares for the White Queen, as follows:
White is simply better.

Thanks @BoredToTears, that was incredibly helpful!
@TheRealPhoenix: While the black's light bishop is technically a bad bishop, it can be very powerful on the b1-h7 diagonal outside the pawn chain. If not traded it generally winds up sitting on h7 behind a pawn, in front of the castled king.

Black's bishop isn't bad because it's outside the pawn chain. It's also going to be more useful than a bishop on e2 so I guess it's good for White to swap.
I think White usually attacks the bishop on f5 hoping to force Black to make some concessions on the kingside before swapping the bishops. By pushing the bishop to g6 and provoking the move ...h6 first, the move Bd3 would at least force Black to capture on d3 after having made a move that doesn't help them.
4.Bd3 can't possibly be a mistake. It's just not the most active way of doing things.

I think it's a little superficial to ignore the analysis and experience of all the world's best grandmasters, who mostly avoid playing 4.Bd3 Does anyone really think that they have all overlooked this marvelous move, or failed to evaluate it correctly? 4.Bd3 is a rarely played move at the GM level, and there's a reason for that. You can't always go by database stats, but it's worthwhile noting that in the Chessbase Online database, 4.Bd3 scores around 43% for White
All moves that aren't popular aren't bad. 4.Bd3 isn't considered the most accurate move and Black doesn't have trouble with it. That's why they don't play it. There's nothing wrong with it.

Whenever I see this when playing black I always trade. The light squared bishop for white is a great attacking piece when black castles kingside ( f7 and h7 targets) so offering to trade it is bad. Black's light squared bishop doesn't have the same advantages and is the bad bishop ( pawns on light squares). After the trade black has a comfortable bishop which can easily travel through the pawn chain and white has a bad bishop left that sometimes becomes blocked.

Whenever I see this when playing black I always trade. The light squared bishop for white is a great attacking piece when black castles kingside ( f7 and h7 targets) so offering to trade it is bad. Black's light squared bishop doesn't have the same advantages and is the bad bishop ( pawns on light squares). After the trade black has a comfortable bishop which can easily travel through the pawn chain and white has a bad bishop left that sometimes becomes blocked.
Thanks for the tip!
Just finished a great game against a strong (relative to me) opponent where we both played very accurately for most of the game. (Full annotated game is on my blog if you wan't to see it.) But there was one move in the opening that has me confused, and I hope one of you can help!
Bd3 isn't an odd move in the advanced Karo-Cann but it is odd to come on move 4. The chess.com's game explorer database lists this move as the 9th most popular, and leads to 42.1% win for black and only 24.3% for white. 365Chess's opening database gives comparable numbers.
This leads me to believe there must be something quite wrong with that move. However I can't find any trap to set, weakness to exploit or even tempo to gain. Anyone know an issue with it?
Or perhaps the disadvantage is very minor. And cause and effect run differently. That is it is a move order that is generally played by people who don't know Caro-Kann very well. And they don't win as often because they don't know Caro-Kann, and the move is just a symptom of the problem.