As I stated much earlier, "sport" is a cultural issue. The International Olympic Committee considers a competition with a single international organization that controls the rules and competition to be a sport. By that definition, scrabble could be a sport if all nations agreed on a single dictionary to use.
Whether chess is a sport or "just" a game depends entirely on the definition used to define sport. Most definitions of sport have no clear amount of physical activity or dexterity required. Is playing darts badly still a sport? Competition chess probably burns about the same amount of energy, but over a longer period.
Well, I read Karpov lose around 20 lbs against Kortchnoï in their 1978 World Championship Match. Of course there are many factors like stress and fatigue who was present, but with the subject of "chess is a sport?" wanted to highlight this little point...