How to deal with Annoying Knights???


You have to try and work out:
1/ If you should allow it by your previous moves....
2/ If you do decide to allow it, you may have judged that you can remove it.. In this position you might consider nd7 of bd6 to challenge the knight and threaten to take. I would judge black is better because after the swap off white has three pawns in the centre on the same colour as his remaining bishop. White has unwisely given up his white squared bishop which is often his best minor piece with this type of pawn structure.

@ Zugzwang:
Thanks for replying. What course of action would you advise taking if white follows up with Nd2, Nf3, and Ne5 replacing the knight I exchanged off?

Now you're looking ahead and thinking like a chess player ! Well after Nd7 and your suggestion of Nd2, after black plays Nxe5 white just hasn't had time to support his knight; there's a pawn on the outpost square. There are of course other variations from that position.
So a knight outpost is a strategic aspect of the game. Generally the question is "can the outpost piece remain there". Then we are into calculating variations and the forcing variations called tactics. Strategy and tactics intertwine. The calculation of variations is being there in that particular position; strategy is informed by experience and some book knowledge that we have gained and can apply to the task in hand.
You have already that a knight outpost is a nuisance. You too can now be nuisance with your own knights on outpost squares !

Place your own knights on even more annoying squares
Also, think about the relative strength of pieces in the current position. In your example, your dark square bishop really doesn’t have any future to speak of due to white’s pawns; if you are able to exhange one of your own weak minor pieces for the other person’s strongest piece, that will always logically be a huge improvement to your position. Thinking about the role / strength of minor pieces and deciding when to exchange and when to avoid it is a huge ganechanger for your game level and one that many beginners have a difficult time wrapping their mind around, I am still working on it