A Primer on Statistics to Help Quell Your Outrage at the Google Memo

By now you’ll probably know that Google employee James Damore recently published a memo regarding differences in population averages in some traits in men and women, questioning the basis for certain diversity practices and indicting a culture of ideological conformity at the tech giant. The result was incandescent outrage, denouncement of the memo, demonization of Damore, widespread “progressive” criticism of science, potentially the most remarkable mischaracterization campaign in recent journalistic history (quite a feat), and, perhaps least interestingly, Damore’s firing from Google. The most important story about Damore’s memo really comes down to the should-be-unbelievable reaction to it, but the topic has also spurred significant interest in a number of related questions.

One is about statistical reasoning and whether or not Damore crossed any lines into sexism or racism. A person could be forgiven for believing that if journalists and commentators fully understood statistical distributions and what they represent that there would have been less outrage. We shouldn’t be so sure, however, that knowledge or understanding has anything to do with the reaction, because Damore was actually exceptionally careful and clear. Still, for those interested, hopefully I can say a few interesting words about the topic.

To oversimplify, a statistical distribution is a mathematical function describing (or, if you’re looking at its graph, showing a picture of) how many individuals within a population we are likely to find at any given point along a spectrum. The famous “bell curve,” which refers to the normal distribution, is a familiar example. 

The normal distribution tells us that if we look at the spectrum in intervals near the average, we’ll find more people than we will further away from the average, and we’ll find some, but very few, people relatively far from the average. Of particular note, the whole point of a distribution is that most individuals aren’t average, and statistical tools like variance and standard deviation exist to account for that. Thus, statistical reasoning doesn’t tell us about specific individuals; it allows us to make specific guesses about traits individuals from the population may exhibit and to estimate rather precisely how likely we are to be right or wrong in those guesses.

For example, the facts of the distribution might be such that about 30% of the population falls in the interval between the average and one standard deviation above the average (standard deviation is one common way to measure the average amount by which people deviate from being average). If that’s the case, given individual A from the population, if we guess that A scores between average and one standard deviation above average for a given trait, knowing nothing else about A, we know we’ll be right roughly 30% of the time and wrong 70% of the time.

When there are two distinct sub-populations within a population, say, men and women within human beings, each distribution can be considered separately. Sometimes those distributions are different, either having different overall shapes, different degrees of variance, or different averages. In this case, we can look at various intervals and determine guesses about the expected ratios of the two populations within a certain range for a given trait, and we can have some estimates based upon the statistical information we have about how often we’ll be right or wrong about any randomly selected individual from either population.

For example, suppose we examine a trait that appears to exhibit statistical differences between men and women, as Damore did. One of Damore’s examples comes down to expected averages in coding capability, which is probably correlated with IQ. Men and women have roughly the same average IQ, but the spreads of their IQ distributions may not be the same (the research is contested). If it is the case, women (for reasons rather neatly explained by the central value theorem from mathematics combined with facts about our evolutionary heritage) have the same average IQ but lower IQ variance than men do. Put another way, there may be more men at the extreme ranges of IQ than there are women, both smarter and dumber, and the numbers show that there are roughly twice as many men as women in each of the top and bottom 2% of IQ.

Is this relevant to Damore’s point? Maybe. His claim is that one of the reasons potentially contributing to there being more male than female software engineers at Google (and more broadly in tech) is that there is a smaller pool of relevantly talented women to pull from. If it is the case that Google only hired from the top 2% of IQ (it certainly isn’t; they aren’t that elite), then there would be a two-to-one male-to-female ratio in potential applicants to Google as determined by IQ score and its relevant correlates, but as the entire tech industry hires only a small fraction of the population, even this fact might or might not impact the sex ratio in tech (because there are enough highly intelligent people to fill spots in many fields). As it stands, Google (and tech more broadly) certainly hires a wider variety of people than just the top 2% in IQ, so IQ variance differences in men and women may only account for a very small percentage (though not necessarily zero) of the sex ratio at Google and in tech. Still, pointing out this statistical difference as a potential variable falls pretty far from constituting anything like sexism.

More relevant than IQ to relevant capability is relevant interest. Damore makes this point also, arguing that on average men and women have differing levels of interest in systems (men) and people (women). There are (even if they aren’t known) distributions for interest in working in tech also, and there are some very good reasons to believe that these show a remarkably sexed difference in average levels of interest between men and women (one of these is that highly intelligent women tend to be more broadly talented than highly intelligent men and thus have more good options available to them, spreading out their degree of interest more widely than for men, but I think we’re supposed to ignore that because it doesn’t fit the discrimination narrative at all). If such differences in interest exist, why they exist is a different question worth investigating carefully with the best tools we have. Yelling the question off the table for being offensive is not among those effective tools.

If there is such a difference, and it is significant, this variable could dramatically skew the potential hiring pool for Google and tech more broadly without the problem having anything to do with discrimination. (Indeed, in this case, it’s the opposite of a problem because such a state represents an increase in fulfillment of individual liberty and thus with life satisfaction for women and men alike, with women standing more to gain due to lingering historical imbalances.) Simply enough, if there is significantly lower interest, it would naturally imply dramatically fewer applicants and thus far fewer hires. Google’s very expensive diversity initiatives seek to correct for this problem, and that they clearly aren’t working effectively makes the question about differences in interest more interesting, not less.

What does this tell us about the individual women working at Google, many of whom were insulted by the memo’s alleged implications? Not much, and probably nothing. All such a difference would tell us is that if we examined an interval of the general population’s tech-interest distribution, at the relevant high-end range, there are more men than women to interview and potentially hire. Okay, so what? What does this say about women working in software engineering at Google? Nothing except that they are in the relevant interval.

Any gendered difference in interest in tech doesn’t apply to anyone working as a software engineer at Google because, whatever the sex ratio in the relevant interval describing high enough interest may be, everyone working in tech at a firm like Google is in it. Thus, Damore’s memo almost certainly neither said nor implied that any woman working at Google isn’t good enough to be working at Google. His goal was to offer potential explanations for why there are so many fewer women working at Google than there are, which seems to be an attempt to resolve a diversity problem, not to exacerbate it.

Swinging to the bigger picture, one point we should take away from this cultural firestorm is that it didn’t make sense and thus reflected something ugly going on. Not only aren’t men and women reducible to averages (and no statistical analysis would say anything like that they are), men and women aren’t reducible to a small number of traits either. Men and women possess many traits, and all the evidence we have from the most gender egalitarian societies on the planet appears to point to a simple fact: individuals are different from one another, and when allowed to follow their differences freely, they will express them.

Perhaps there are genuinely more men than women interested in coding, in which case it’s probably better that more men than women work in software engineering because when they do it frees up more people to pursue their own goals and work in fields they enjoy. Perhaps there are more women than men interested in medicine and public administration, in which case it’s probably better that more women than men work in those fields because that allows more people do something they most like to do. Maybe those facts can change with time, culture, or educational initiatives, one way or another. Our diversity initiatives would do best to reflect those realities, whatever they are, rather than fighting them, but then, that’s what James Damore said.