What is wrong with this opening?

In a blitz game, I had played 1. e4 e5 2. Nf3 Nc6 3. Bc4 Bc5 4. 0-0 d6 and I had premoved 5. d4-- 

I did lose the game but was wondering if d4 was such a radical notion after all. As a gambit, I believe it gained me good tempo and initiative, and was wondering if their was any real analysis and if there was even a refutation. If 5. ... Nxd4, 6. b4 is strong which can go in these sequences: (b4) Bxb4 7. Nxd4 exd4 8. Qxd4 with strong play on the kingside for the pawn; (b4) Nxf3 7. Qxf3 threatening mate on f7 while attacking the bishop as well; (b4) Bb6 7. Bb2 giving white fluidity on d4 and strong tension/ play.  So maybe Nxd4 wasn't a good move after all?? If 5. ... exd4 6. b4 Bxb4 (seems better than Nxb4) 7. Nxd4 and this looks great for white-- I will place my dark-squared bishop on b2 and my other knight either on c3 or e2, followed with Re1 and wherever the queen fits in.  If instead 5. ... Bxd4 6. Nxd4 Nxd4 and 7. f4 looks just fine. Please give your thoughts about this and reply/ respond if you see any error in my calculations. Thanks!

the main idea of b4 was to allow good piece placement, such as 7.Bb2

Fluidity as in chances for good initiation and tempo later in with things like Ne2, Qg3 or Qb3, Kh1, and opening the center with f4-- and if your opponent castled kingside, you  have amazing bishops staring right down from one side to the center and to the corner.  After analyzing with Fritz, It says you have compensation once the position is open, and it is obvious all gambits played need compensation.  If the opponent castles queenside, you already have the lovely b4 set in place and you get a4 with tempo against the black-squared bishop. If you think White has no compensation, please show me some good analysis/ proof.

That just hurts white's position, taking c3 away from the undeveloped knight and not making an effort to bring out your pieces and get compensation at all.

If you want Houdini's judgment, I'll provide it, though it is likely to condemn this prospect if both an IM and CM feel that it isn't worth the pawn . . .

I ran this through Houdini, the Lord over All Accurate Play, it apparently believes there is some extremely minor compensation for the pawn -- meaning, White isn't just a pawn down. However, Black does have the advantage. This is the analysis, reached at a depth of 21 half-moves, 450 million positions analyzed, and 2.5 minutes of thinking: 

And now I actually know why! THANK YOU