There aren't people who play white all the time. Maybe you meant to say people who play d4 as white all the time.
On topic I agree. The Budapest is dangerous because there is no way to dodge the main lines like in other openings e.g anti sicilians.
I think 3)Nf3 exd4 Nxd4 is the simplest anti line with e4 planned after Nc3 but ...c5 maye cause problems(but with Nb3 and N-c3,d5 white looks to have a sicilian esq postion but with less central control {c5 pawn not e5})
Thats all IMO though!