I wouldn't say so because of the sheer growth that chess has had recently. You could find that many people play chess (i.e. 100 Million Chess.com users) and that doesn't even include those who know how to play/had chess integrated into their childhood which is quite common.
Chess is often played individually, and it's a game that rewards solitary study and practice. This can lead to the perception of chess players as introverted or independent thinkers, which is sometimes associated with "nerd" or "geek" culture.