Its something you get over, i remember back when i started and was playing the dragon, i was always so freaked out by pawn storms i just had no idea what to do, i had basically the same thoughts as you.
However, then I began playing the white side of it for a while, and had my pawn storm initiative blocked again and again and again, so i noticed just how simple it can be to block these, you just need to get enough experience to understand how.
How Come i Lose when Opponents throw their Pawns at me?
In a position, when there is opposite side castle, that wins who can attack the opponents king first, in these positions it is easy to attack the king's. And the typical way to attack the king is to attack with pawns, because you dont weak your king's position, because it is on the other side. You have to close the position where your opponent attack, and open the positon to the opponent's king. As long as you dont move with the pawns which protect your king, you can easily close the position at your king.
Don't move the pawns in front of your king, and do your best to open lines somewhere else on the board. They are double edged positions to play, so if it's in blitz there's certainly plenty of chance to error, and it is likely that whoever get the initiative with the attack will be in a very good position to win the game (on time or checkmate).
It happens a lot of times...i castle kingside he castles queen side, or vice versa..... and they roll down their pawns with a lot of pressure and soon my position is blown up....i do that too, this is mainly in blitz... to me it looks like whoever throws down those pawns first has a better chance of winning, is this true?? or is it just me? that i cant defend my position well enought?