Statistical Analysis on Gender Difference

Everyone knows that the best male players are rated higher than their female counterparts. As of today, the top female player is Judit Polgar (53rd in world ranking) with a rating of 2686, over 100 points lower than the world leader Magnus Carlsen with 2814. Polgar's best rating was 2735 in the middle of the year 2005; the highest male rating achieved was around 2850 by Garry Kasparov. So there appears to be a difference of just over 100 rating points between the sexes.

According to the gender-neutral outlook, this difference is attributed to there being an order of magnitude less female players. Consider separate Gaussian distributions of male and female players: if their peaks are located at the same rating (i.e. men and women have equal skills on average), then the larger amount of male players results in the top of their distribution being located higher than the female distribution. Intuitively, this is like placing a large cup over a smaller cup - the larger cup extends farther in all directions from the center, even though the cups are centered in the same exact location.

Thinking about this last night, it struck me as odd that (to my knowledge) nobody had ever presented any data to back this up. So I spent some time this morning trying to find out the truth behind the matter. I downloaded the entire list of FIDE-registered chess players (273886 players as of today, some of whom are not rated) and wrote a program to process the data into two sets, one for each gender, comprising the distribution of registered ratings. I then made a Gaussian fit to both sets of data, and was surprised by the result.

The male distribution (blue) shows a peak at around 2016 points, while the female distribution (red) shows a peak at around 1920 points. A hasty conclusion would be that men are indeed about 100 points "better" on average than women.

Anyone with a trained eye will see that the data is far from perfect. Up until recently, FIDE did not issue ratings below 2000, which is visible as a glitch in both data sets at around that value. The peaks are "supposed" to be located at around 1200; this does not happen here because average everyday players do not register for a FIDE rating. From a statistical viewpoint, it must be assumed that "2000 is the new 1200", and that the set of players to be analyzed consists only of those players who are good enough for a FIDE rating. Thus, the gender-neutral outlook may assume a new explanation for the inter-gender difference: the (skillwise) threshold for registering with FIDE is actually lower for women than it is for men.

Caveats aside, it is tempting to play with the numbers. If Judit Polgar had a handicap of 100 points in her favor, she would now be about even with Vladimir Kramnik battling for fourth position in the overall list of players; at the apex of her career, she would have been on a par with Garry Kasparov. Since she's the best female player of all time, it all makes a fascinating kind of sense.

Questions? Comments?

Well, I'm glad someone's actually using numbers instead of their "theories". I don't really "like" the idea men would be inherently better, but I can't really back it up. Some possible explanations for the curve:

1. Women are represented way more at the scholastic level, before they hit puberty. So some of those ratings could be of inactive players who never came near their potential.

Counter-argument: How many scholastic players have FIDE ratings?

2. Is a rating really accurate enough? I've heard 50 is the margin of error, which really isn't all that accurate.

Counter-argument: 100 points is quite a lot.

The most compelling argument is that the data is effectively split at around 2000 points in both sets. If FIDE's policy of issuing sub-2000 ratings is too recent, then there is not enough data on the lower side. Indeed, it's clearly visible in the male data set that the Gaussian fit is a sort of compromise between the two sides, being too steep on the higher side and not steep enough on the lower side. As time passes, there should be an increase in the number of lower-rated players, which would move the peaks downward in rating. Arguably, this movement should be similar for both genders, but it's impossible to know for sure.

Puchiko, your first point hits the mark. If there is a gender difference in representation for different skill levels (which is quite probable), it would definitely skew the data. But again, this is something that can only be speculated upon. As for your second point, it's not statistically valid; while an individual rating may have a large margin of error, the collective margin is reduced with an increased amount of data.

Whatever the "ultimate truth" may be, the main point of this little experiment is to show that the current lack of women in the top 50 chess players cannot simply be attributed to there being about ten times more male players, if the registration threshold for a FIDE rating is the same for both genders. If we want to believe that men and women are equally good at chess, then we must assume that women are actually more eager to play seriously (by which I mean to get a FIDE rating) at the lower levels than men - which can make sense if the atmosphere is such that women have something to prove.

The location where computing happens in males' brains is bigger and more developed than the female counterpart. There's significant anatomical differences between the male and female brains, because we developed symbiotically. We complete each other, we were never meant to compete. It's why people now have to come up with theories as to why females are doing worse, so as not to hurt half the population. It's obvious why, open up a biology textbook or go to a science lab. Chances are you'll see that over 70% of science phds in your local lab are male. 

Would you honestly argue that Polgar stands a chance against Carlsen as of now?

Polgar had a few victories against the top players, but these were rarities. She lost very often to the top ten players.

Kosteniuk is another strong WGM, but she has trouble against players over 2700.

It just seems like men are better developed for chess.

"Boys generally demonstrate superiority over female peers in areas of the brain involved in math and geometry. These areas of the brain mature about four years earlier in boys than in girls, according to a recent study that measured brain development in more than 500 children. Researchers concluded that when it comes to math, the brain of a 12-year-old girl resembles that of an 8-year-old boy. Conversely, the same researchers found that areas of the brain involved in language and fine motor skills (such as handwriting) mature about six years earlier in girls than in boys."

--WebMD

On the plus side, women are better when it comes to language, which is a bonus.

Not another women-in-chess thread.

In her prime, Polgar might have been a tough match for Carlsen. But it's unlikely she'll ever reach that level again.

Data is one thing. The conclusion about (genetic, to be precise) ability another.

What if (and I claim this!) , in average, women stop earlier than men to 'take chess seriously', (meaning, women tend not to go to the club more than one time a week, not to spend more than 3 hours a week reading and analysing, they do not sit and spend 6 to 12 hours a week playing chess games on the internet etc., BUT MEN DO) ?

Wouldn't this automatically mean that the expectation value of rating is also necessary higher for men than for women ?

(The statistics above says exactly that). And isn't this just a sign of higher average  dedication of men to chess, but NOT of higher talent ?

I claim that despite of all efforts Mr. Polgar, some Georgian parents, or some Chinese trainers  put into their female 'trainees', they cannot entirely compensate the other persuasions. 'You are not cute when you are angry and competitive.' 'Which man will love you if you are so smart?' 'Can you be there for your baby when you are in a tournament?'

Judith's rating went down after she gave birth.

I'd be interested in the standard deviation of both data sets. I believe, statistically, you can compare the averages of two data sets combined with their standard deviations, and if the averages aren't far enough apart (half a standard deviation? Not sure, been too long), the difference in the average is statistically insignificant. To be able to say that ever a 100 point difference means anything, we need more statistical analysis of this data.

If she needs the handicap in the first place then she's objectively weaker than those above her.  May as well give Carlson a 150 ELO handicap and have him be "on par" with Houdini connected to a supercomputer. 

 I did not deny the significance of the statistics (given by its author as 'rough'). I just wanted to say that one must not jump to an odd conclusion about men being having better 'gift' for chess. 

I agree with schlechter, it is because less women play chess very competitively and more (proportionally) play casually or at a schoolastic level.

Great thread!

No, dude.  He's a parrot.

He has worked his dark arts to revive this dead thread!

Cluck, cluck!

Threads never die.

Women are not worse at chess because they lack ability.  Women are worse at chess because they understand the truth:  playing chess is ultimately a waste of time, and spending all your time with boys/men competing against each other because of testosterone-fueled biological imperatives is just annoying after a while ;)...