Gelfand

Five years ago he came an inch from winning the title match against Anand, I wonder if any other player has been so close to be World Champion and then been outside the top 40 a few years later.

Leko dropped quite a bit after his title match but it took him a bit over ten years to fall below the top 40.

We can only be grateful that Leko, Gelfand and Karjakin didn't win. Otherwise things would just seem wrong.

Of course, Gelfand was "only" 2727 when he played the title match and is just above 2700, so the fall in itself isn't that steep, but his stats against the two best players of the last decades have also been unimpressive. Against Kasparov he was 0-9 with 8 draws, while he won a game against Carlsen when the latter was 15 years old, but after that scored 0-5 with some draws. 

At the same time also Anand had bad results against these two after a good start. Initially he had 2-1 against Kasparov and a huge plus against the young Carlsen. Then Kasparov scored 14-1 in wins in their last 15 decisive, while Carlsen has 9-2 in wins over the last seven years. It's funny in a way that the players in the 2012 title match had such horrible stats against Kasparov + Carlsen, something like 3-37 in wins against them after the initial games.

+1 about Le Quo. How about Khalifman (who?)?

Khalifman was "only" one of the minimatch knockout champions from the split years, and was #44 when winning that title. Then he actually got as high as #12 four years later, but faded out of the top 100 a few years after that. But he was never anywhere close to play for the "real" title.

.

And at the moment it looks like this, quite a few old challengers and FIDE Champions gathered there...