trading K / B

Hi guys. I wanted to have your thoughts on the two positions below. I like to play the gambits, and usually players leave my knights alone and use their next move to develop, but the last days I have met some players who simply trade the knight for their bishop. 

Obviously I can't do much other than to take back with the rook, but other than loosing my ability to castle queenside I don't see the big loss? 

Im I right in assuming that to trade N / B so early is not a good move? After all, the only thing accomplished is that my opponent trades his already developed piece against my undeveloped?

If your opponent decides to trade that Bishop for your knight with no concrete reason then you should be ecstatic.

thanks Zigwurst, I thought so too, it looked so weird that I started to think I was missing something. 

Nice one Fiveofswords, that one I might have missed. Generaly I want my bishop at G2 so that is not a problem for me, but I might be able to use that as an attack at some point.