How should I have played versus this as black? (Caro-Kann)
5 Ng5 wastes a move. Since Black has removed a defender of the d-pawn, you should take the initiaive by attacking it with 5 ... c5 or 5 ... Qb6.
Why not kick the knight with h6 right after Qf3? The knight can only go back to h3 then and is even further from d4 than ever before. The bishop can work nicely with that h7 square too? If he ignores the attack on the knight and plays g4 as planned, you can just cover the mate with Bg6 and the knight will still have to retreat. This is what I'd suggest knowing jack durp about the Caro Kann.
One thing to keep in mind that strange moves sometimes do require different replies than you might be used to. You can't "continue as normal" if your opponent didn't play the "normal" way. What you might be doing instead is miss opportunities to punish subpar play.
Hmmmm maybe kicking the knight with h6 isn't a brilliant then due to the temporary knight sack which then leads to capturing my bishop and leaving the king exposed.
Is this a common line versus the Caro-Kann? If not why isn't it, can any of you maybe post up a scenario where white doesn't have as much advantage?