why can't i en passant?

En passant is only available if your pawn is also on the 5th rank (in this case, if your pawn was on a5 and Black played b7-b5, then en passant would be legal).
I think of it this way: en passant is available if you could have captured the pawn had moved only 1 square as its first advance.
https://www.chess.com/terms/en-passant