En passant

I once got accused of cheating (by a low ranker) and/or expoiting a chess.com bug by capturing en passant. I've also played OTB games with relative newbies who say, prior to the first move: "Can we not play en passant?", to which I reply: "We certainly will play it. It's a rule of chess". What is it about this rule that makes it daunting to the beginner to the extent that it is assumed to be 'negotiable', and not understood by newcomers to the game? 

ED. 

We calls it slidey-pawn on this side. That ooh-la-la frenchy talk just puts people off.

Because it's a rule that most novice chess players first hear of in the form of "No, that's an actual chess rule, honest"

And so they don't want to look like an idiot in the case that they were being tricked.