If the opponent wastes time, then usually it's safe for you to waste time as well. So no, it is not (usually) bad at all. But that does not mean it's the best! :)
Most often, the best moves in the opening involve getting your knights and bishops off the back rank as quickly as possible and castled to safety. If you can afford to moves like this (your opponent has no threat), then it's usually best to do so.
However in my own experience, sometimes my opponent plays a very odd and passive move that I don't know the best response. In those cases I know I have more freedom of choice, so I may respond with an equally passive move in an attempt to steer the game into a position where I'm more comfortable... but again, this is not usually the best move.
Is it bad for me to move the same piece twice in the opening if opponent moved his twice?