Best inactive World Chess Champion

In 1859, Paul Morphy was generally regarded as World Chess Champion... though he did not play a match against Staunton.

After 1859, Paul Morphy did not play serious chess. The estimates are that he played maybe 75 games total, including casual games, between 1859 and 1869, and no games with anyone at all 1869-1884.

Yet by being alive and declining to play any challengers, Morphy remained the world chess champion. Steinitz was clearly the best active chess player in the world through 1860s and 1870s, but did not claim to be World Chess Champion. Because he could not, because Morphy was alive but would not play. It was only after Morphy died in 1884 that Steinitz and Zukertort could play for Championship.

Since Morphy, Fischer and Carlsen have stayed alive but declined to play serious challengers.

Who has been the best inactive World Chess Champion - Morphy, Fischer or Carlsen?