Your assertion that white has lost tempo after ...Bf5 is unfounded. White simply plays 5. Ng3, forcing your bishop to retreat and allowing him to continue his development unhindered.
Nevertheless, it is a decent line- just not for the reason you described.
Instead of 5. ... Bd7, you might want to look into Nc6. If White captures, you have a supported center pawn and if he doesn't, you have the threat of Bg4 coming soon. No big deal though, since you can now play e6 without worrying about hindering your light-squared Bishop.
You note that White is lacking a pawn presence in your game, which is good, but you don't take advantage of this. The move 7. ... e5 takes control of the center and threatens both e4 and d4. This puts White behind a big tempo (the Bishop?) and center control.
The rest of the game seems like you were playing a little too quickly and lost track of White's Queen 
. I like the determination and the "go for the throat" checkmate you made at the end.
The Caro-Kann Defense is an opening for black against an e4 pawn opening. The Caro-Kann has been used by many GMs including former World Champion, Anatoly Karpov. The defense has been recognized to provide enhanced pawn formations for black into the endgame, which can provide a superior outcome.
[under construction]