Just promote your pawn(s) to a queen.
Can't win without the queen?
Easier said than done :=D
A good chess player should easily win a chess game without the queen. That's the purpose of this. Trying to devolop a more "creative" way to pla.

Try playing some games where before move 1, you (and your opponent) take your queens off the board. Look into minor piece endgames too.
It's good that you noticed your reliance--I always used to trade queens just to put my opponent in your position, because they never knew what to do without the queen. Realize also that once the queens are gone, often you go to an endgame. It -is- much harder to checkmate without a queen--you just need to be good enough with your other pieces to make a new one.

Easier said than done :=D
A good chess player should easily win a chess game without the queen. That's the purpose of this. Trying to devolop a more "creative" way to pla.
Easily winning without a queen? Not always. Promotion is the main theme of most endgames for a reason. You shouldn't have any problem playing once queens are gone, and should be able to win an endgame, but sometimes (oftentimes) "winning" the endgame means promoting first, and using the queen to checkmate or win more material.

Learn the K+R vs K checkmate. Learn how to win / draw king + pawn vs king, then how to use a pawn majority e.g. K+4 pawns vs K+3pawns with all pawns on one side of the board.
But getting a basic endgame book would be even better (would cover this stuff and more).
-----------------
But as others said, most endgames are resigned by one player because the threat of queen promotion is unstoppable. If what you want is to learn some mates not involving a queen, then you could get a book like Reinfield's 1001 brilliant ways to checkmate (or similar books).

I've often noticed that players around my level of ~1300 tend to be afraid to trade queens. I think way too many players at my level place way too much emphasis on the queen and some tend to not know what to do when she's gone.

dont trade queens unless:
a) it is forced, or else u gain a disadvantage
b) You are winning in material, or in position
I tend not to launch an attack without a queen, unless the position is in a way that it is fine to.

Have a look at some standard mate themes, that would probably give you a few ideas. Batgirl did a nice thread on the subject, but I'm not sure what the link is. For me, it really helped looking at pictures of different types of mates (no, I'm not talking about the ones on russianmailorderbrides.com, no).
Also you could issue OTB challenges with you being a queen down. Might jog an attacking idea or two when you're under more pressure.

I almost always look for a way to exchange queens because i find a combination of bishops and knights are incredibly tricky to counter and hard to spot, especially when there are still a lot of pieces and the board for them to "hid" among. Try practicing playing without your queen against some people you know who will start easy on you then get progressivly harder as you get better. I did that with bishops and it helped me a lot because i would always lose once my bishops were gone. I hope this helps any!
I noticed that i have this problem.
Let's say for example me and my opponent exhange queens. And i take the lead after that. I wont be able to checkmate him without the queen. I rely way to much on the queen, how do i improve this?