I think that if Fischer were a "coward" he would not have granted Spassky an unsanctioned rematch after he beat Spassky in the WCC.
Kasparov has written in an annotated WCC game that he himself was afraid of Karpov. However, Kasparov still defeated im.
If an athlete chooses to retire when they are in their prime, instead of fading away, that doesn't mean they are a coward. There is nothing wrong with quitting after you have accomplished your goal. Eventually Carlsen will quit, and sure enough, the haters will call him a coward too.
Why the heck are you people arguing about this? There's no potential benefit. You aren't these people, I'm under the impression none of you are related to them, and likely none of you stand to gain or lose anything based on the ambiguous outcome. Let your passion shine elsewhere please, where you're less likely to offend somebody, or embarrass yourself.