Any pawn that makes it across the board can be promoted. There is no limit on how many times you can do this in a single game.
Advancing pawns

As a courtesy Ive always exchanged for pieces that are off the board. I dont think you should promote to another queen unless it forces mate.

Yup, I usually do this in quick live chess games when my opponent refuses to resign...just keep on promoting all the pawns I have left
It is possible to advance multiple pawns and exchange them for multiple queens, isn't it?