Need approach to handle out of book moves

Use brain and think outside of book. Evaluate and be creative 🐛 That is chess. Other thing you mention is called memorizing 🤓

When they take you out of book then you use your book knowledge to figure out why the move they made is not a book move.

A very brief discussion of 1 e4 c5 2 Nf3 d6 3 Bc4 Nf6 4 d3 can be found in Starting Out: The Sicilian by GM John Emms (2009).
https://web.archive.org/web/20140627122350/http://www.chesscafe.com/text/hansen123.pdf

How do you find moves in a non-opening position?  Hopefully you have a guiding plan, even if it is something as simple as hack the enemy King.  Then you look for possibilities and candidate moves, then you calculate to see if it is safe, then you play it.  And you know what?  This works for the opening as well.

If you are struggling with non-book moves on move 3, that suggests you do not understand the openings you are playing.  Do you understand the ideas behind the Sicilian?  We are taught to develop quickly and to not make any unnecessary pawn moves.  Why, then, does Black play 1...c5, which doesn't help developing?  Why does Black make four pawn moves in the first five moves?  If you cannot explain, with words, what Black is doing, then you likely shouldn't play it.

Too many players cling to book moves like some unfailing safety net, when it's fully possible for theory to be wrong for you.  In my case, I don't understand the Closed Ruy Lopez.  I've tried books, games, videos, but nothing has clicked yet.  I only see a blocked position with little chances for either side.  I don't know what to do next.  It doesn't matter if the book says a certain line gives White an advantage, because even if it does, when theory ends I still won't know what to do, and I'll surely blow it.

Someday I will understand the Closed Ruy and confidently play it.  Until then, I play moves that I do understand.  Theory says it is harmless, but I know what to do afterwards, and that's infinitely more useful.

When I started playing online I was at the beginner level and I was doing fairly well. With that in mind, I figured I could do even better if I knew some openings. So with Black I started playing the KID and instead of doing better I was getting killed. I tried different book defenses and got the same results. Somewhere along the line it struck me that since I was a beginner, my opponents probably knew the KID better than I did. And another thing I noticed was that most of them weren't playing the KID the way they were supposed to. They would make moves that weren't covered in the books. So I turned to databases and found that some of those moves weren't found there either. I was on my own. And that's where your opponents want you to be.
Chess is mostly tactics. The purpose of the opening is to reach the middlegame, hopefully with equality or better. The KID is buried under a mountain of theory. And if the KID is a mountain, the Sicilians are the Himalayas. I honestly feel that beginners will choose the Sicilian simply because it has a strong reputation. And also because it will be a long time before you actually have to start thinking on your own.
Every opening is a path up the mountain, and there are many. And which is best is going to depend on you. You should play an opening which makes sense to you, and which you feel natural with. Stick to it so you can focus on tactics and endgame, which is where games are won. Later you will naturally want to try other openings.
Chess is a complicated game and your question does not have a simple answer. You have to figure it out and learn from your mistakes. 

"You may learn much more from a game you lose than from a game you win. You will have to lose hundreds of games before becoming a good player."
--- Capablanca

buy a new book?

And you have just fallen for the biggest mistake in all of chess.

DO NOT MEMORIZE!!!  UNDERSTAND!!!

I can easily memorize 1.d4 Nf6 2.c4 g6 3.Nc3 d5 4.cxd5 Nxd5 5.e4 Nxc3 6.bxc3 Bg7 7.Bc4 c5 8.Ne2 Nc6 9.Be3 O-O 10.O-O Bg4 11.f3 Na5.  No sweat.  Could play it blindfolded.

That said, such memorization is USELESS!  Ok, White now plays a move.  That could be a Bishop Retreat, or it could be 12.Bxf7+.  It doesn't matter.  The big picture of what Black is trying to achieve makes no sense to me.  This is NOT an appropriate opening for me because I seriously do not understand the Grunfeld.  The few rare times I face it as White, being primarily an e4 player with occasional d4 play, I don't play the Exchange.  I play the Russian Variation.

That said, since the Exchange Grunfeld to me doesn't make sense, the middle game ideas that is, I don't play the Grunfeld.  I instead play the King's Indian Defense because I UNDERSTAND Black's ideas.  Everything from the Kingside Pawn storm in the Classical Mar Del Plata to the various piece sacrifices (...Bxh3, surrendering the Rook on a8 to buy time, etc) to positional lines like the Fianchetto King's Indian or the line I play against the Saemisch, defusing White's center.

When White deviates or I get through the opening phase, I know the ideas of what to do next.

This is what you need to be doing when searching for an opening.  UNDERSTAND the opening.  Don't memorize it!  I can memorize the Grunfeld and Dragon Sicilian.  What I understand are the King's Indian and French.  Therefore, I play the latter, not the former.

If you don't understand what to do when your opponent goes out of book, you don't understand the Opening you are playing and have absolutely no business playing it!

I like your responses to Bc4.  You played it the same way I do.  a6 is less accurate after the bishop is already developed IMO.  You do have to watch out for Ng5 but if they play it you can then play e6 and why is there a knight on g5?  He's out of place and white has lost a tempo.  That's why i like playing Nf6 first... if Ng5 then e6!  In the Sicilian too I often just try to play d5 as soon as possible.  So if white allows it I'll get d5 in very early and its a computer approved plan for the most part and I like the positions that arise.

When people play out of book though just stick to your opening principles.  I like to think of the opening as a "race" to develop and castle.  You also want to fight for the center and determine the pawn structure you want.  I mean, I like to memorize and play book moves too, bc I like having traps and ideas in mind, and spend very little time on them early on, but once the game goes out of book it can be really enjoyable to figure out the best places for your pieces.  When in doubt though just develop and castle asap!  And I mean in terms of tempo but also in terms of the clock too!  I feel like the opening is probably the least important part of the game so I try to spend very little time there and reserve my clock for middle game and end game and especially critical positions.

So those are my thoughts but I'm only 1500!  So you should probably listen to the title players if they comment.   

get a new book?