To do so willy-nilly is not good, yes.
should i exchange queens early

Seek the advantage, in whatever form you can achieve it. If a queen trade gives you the advantage in one game, trade. If in the next it does not, then don't.
Let your opponent be the one to come up with inflexible gameplans he thinks of before the game even begins, and follows blindly. You be the one who adapts to the needs of the position, and stands ready to instantly seize any opportunity, the moment it presents itself.

Pieces aren't often worth the same amount. Also there is often a change in the position after an exchange like altering pawn structure or tempo.
So exchange when it's good for you, and don't exchange when it's bad for you.
Simple right? Yeah, not really. But try to make your best guess. Trading pieces for non-chess reasons like deciding before the game you don't like queens isn't a good way to play.
beginner question
in my last couple games i been trying to exchange queens as early as i can, i just dont like dealing with them, is that a bad idea? to exchange queens early,