I prefer 6 h3 against this gambit, getting rid of the Bishop's pin.
8 c3 is weak.
Your opponent's 9 .... Bb4 is horrendous, giving up a tempo.
11 Qa4 is sharper, as your e pawn is taboo for the moment.
14 Ne4 was more to the point.
18 Nc4 isn't a good move, because Black can play 18...Qc5+ (which fortunately he/she missed), forcing the Knight back to e3, where it is pinned. 18 Ne4, followed in most cases by Bf4 is better.
20 Kg3 is pointless. The King should stay at f3 for the time being, to answer Rc3+ with Be3. Better for White is such as 20 Be3 or Qe4.
22 Ne3 will hold the position, although Black has slightly the best of it.
Your suggestion of 23 b3 is met by Qe4.
25 Kh2?? is a horrible move, which should be met by 25... Rh3+, winning your Queen for Black's Rook.
The two main things that you did wrong were to neglect your development and to not be aggressive enough when necessary.
This is one of my games. Take a look if you like and tell what I could have done better or any other remarks you want to make.Between () I have put an alternative move. I think this is a better move. I would like to hear your opinion.