Often it's not possible to hold it at bay with only defensive moves. That sort of play is sometimes just a fundamental part of the position for your opponent. In fact in such situations trying to play on "their" side of the board will often help the opponent.
What you do is create counterplay. Unless you completely screwed up the opening, there will be fundamental things for you too. Play a pawn break in the center or kingside and make your pieces as active as you can in that area.
Even when this turns out badly, it will buy you some time, and create at least a puncher's chance. In contrast when you defend passively, your opponent has a free hand to build the attack forever.
So say in some variations of the French, Sicilian etc where white castles queen side how do you guys deal with aggressive pawn storms from black? I usually have trouble holding them at bay :/