Or go the other way and ban pawn double push.
En passant
Being upset at en passant or the double pawn move is just as silly as being upset that knights or bishops move the way they do. They're both well established rules of the game, and they have been for hundreds of years. People are quick to complain when they're losing, but if those moves benefited them in a certain situation they would be more than happy to play them.
En passant should be more widely accepted. People are being bad sports and that is their fault.