A Primer on Statistics to Help Quell Your Outrage at the Google Memo

By now you’ll probably know that Google employee James Damore recently published a memo regarding differences in population averages in some traits in men and women, questioning the basis for certain diversity practices and indicting a culture of ideological conformity at the tech giant. The result was incandescent outrage, denouncement of the memo, demonization of Damore, widespread “progressive” criticism of science, potentially the most remarkable mischaracterization campaign in recent journalistic history (quite a feat), and, perhaps least interestingly, Damore’s firing from Google. The most important story about Damore’s memo really comes down to the should-be-unbelievable reaction to it, but the topic has also spurred significant interest in a number of related questions.

One is about statistical reasoning and whether or not Damore crossed any lines into sexism or racism. A person could be forgiven for believing that if journalists and commentators fully understood statistical distributions and what they represent that there would have been less outrage. We shouldn’t be so sure, however, that knowledge or understanding has anything to do with the reaction, because Damore was actually exceptionally careful and clear. Still, for those interested, hopefully I can say a few interesting words about the topic.

To oversimplify, a statistical distribution is a mathematical function describing (or, if you’re looking at its graph, showing a picture of) how many individuals within a population we are likely to find at any given point along a spectrum. The famous “bell curve,” which refers to the normal distribution, is a familiar example. 

Empirical_Rule.png
For the normal distribution, the values less than one standard deviation away from the mean account for 68.27% of the set; while two standard deviations away from the mean account for 95.45%; and three standard deviations account for 99.73%.

The normal distribution tells us that if we look at the spectrum in intervals near the average, we’ll find more people than we will further away from the average, and we’ll find some, but very few, people relatively far from the average. Of particular note, the whole point of a distribution is that most individuals aren’t average, and statistical tools like variance and standard deviation exist to account for that. Thus, statistical reasoning doesn’t tell us about specific individuals; it allows us to make specific guesses about traits individuals from the population may exhibit and to estimate rather precisely how likely we are to be right or wrong in those guesses.

For example, the facts of the distribution might be such that about 30% of the population falls in the interval between the average and one standard deviation above the average (standard deviation is one common way to measure the average amount by which people deviate from being average). If that’s the case, given individual A from the population, if we guess that A scores between average and one standard deviation above average for a given trait, knowing nothing else about A, we know we’ll be right roughly 30% of the time and wrong 70% of the time.

When there are two distinct sub-populations within a population, say, men and women within human beings, each distribution can be considered separately. Sometimes those distributions are different, either having different overall shapes, different degrees of variance, or different averages. In this case, we can look at various intervals and determine guesses about the expected ratios of the two populations within a certain range for a given trait, and we can have some estimates based upon the statistical information we have about how often we’ll be right or wrong about any randomly selected individual from either population.

For example, suppose we examine a trait that appears to exhibit statistical differences between men and women, as Damore did. One of Damore’s examples comes down to expected averages in coding capability, which is probably correlated with IQ. Men and women have roughly the same average IQ, but the spreads of their IQ distributions may not be the same (the research is contested). If it is the case, women (for reasons rather neatly explained by the central value theorem from mathematics combined with facts about our evolutionary heritage) have the same average IQ but lower IQ variance than men do. Put another way, there may be more men at the extreme ranges of IQ than there are women, both smarter and dumber, and the numbers show that there are roughly twice as many men as women in each of the top and bottom 2% of IQ.

Is this relevant to Damore’s point? Maybe. His claim is that one of the reasons potentially contributing to there being more male than female software engineers at Google (and more broadly in tech) is that there is a smaller pool of relevantly talented women to pull from. If it is the case that Google only hired from the top 2% of IQ (it certainly isn’t; they aren’t that elite), then there would be a two-to-one male-to-female ratio in potential applicants to Google as determined by IQ score and its relevant correlates, but as the entire tech industry hires only a small fraction of the population, even this fact might or might not impact the sex ratio in tech (because there are enough highly intelligent people to fill spots in many fields). As it stands, Google (and tech more broadly) certainly hires a wider variety of people than just the top 2% in IQ, so IQ variance differences in men and women may only account for a very small percentage (though not necessarily zero) of the sex ratio at Google and in tech. Still, pointing out this statistical difference as a potential variable falls pretty far from constituting anything like sexism.

More relevant than IQ to relevant capability is relevant interest. Damore makes this point also, arguing that on average men and women have differing levels of interest in systems (men) and people (women). There are (even if they aren’t known) distributions for interest in working in tech also, and there are some very good reasons to believe that these show a remarkably sexed difference in average levels of interest between men and women (one of these is that highly intelligent women tend to be more broadly talented than highly intelligent men and thus have more good options available to them, spreading out their degree of interest more widely than for men, but I think we’re supposed to ignore that because it doesn’t fit the discrimination narrative at all). If such differences in interest exist, why they exist is a different question worth investigating carefully with the best tools we have. Yelling the question off the table for being offensive is not among those effective tools.

If there is such a difference, and it is significant, this variable could dramatically skew the potential hiring pool for Google and tech more broadly without the problem having anything to do with discrimination. (Indeed, in this case, it’s the opposite of a problem because such a state represents an increase in fulfillment of individual liberty and thus with life satisfaction for women and men alike, with women standing more to gain due to lingering historical imbalances.) Simply enough, if there is significantly lower interest, it would naturally imply dramatically fewer applicants and thus far fewer hires. Google’s very expensive diversity initiatives seek to correct for this problem, and that they clearly aren’t working effectively makes the question about differences in interest more interesting, not less.

What does this tell us about the individual women working at Google, many of whom were insulted by the memo’s alleged implications? Not much, and probably nothing. All such a difference would tell us is that if we examined an interval of the general population’s tech-interest distribution, at the relevant high-end range, there are more men than women to interview and potentially hire. Okay, so what? What does this say about women working in software engineering at Google? Nothing except that they are in the relevant interval.

Any gendered difference in interest in tech doesn’t apply to anyone working as a software engineer at Google because, whatever the sex ratio in the relevant interval describing high enough interest may be, everyone working in tech at a firm like Google is in it. Thus, Damore’s memo almost certainly neither said nor implied that any woman working at Google isn’t good enough to be working at Google. His goal was to offer potential explanations for why there are so many fewer women working at Google than there are, which seems to be an attempt to resolve a diversity problem, not to exacerbate it.

Swinging to the bigger picture, one point we should take away from this cultural firestorm is that it didn’t make sense and thus reflected something ugly going on. Not only aren’t men and women reducible to averages (and no statistical analysis would say anything like that they are), men and women aren’t reducible to a small number of traits either. Men and women possess many traits, and all the evidence we have from the most gender egalitarian societies on the planet appears to point to a simple fact: individuals are different from one another, and when allowed to follow their differences freely, they will express them.

Perhaps there are genuinely more men than women interested in coding, in which case it’s probably better that more men than women work in software engineering because when they do it frees up more people to pursue their own goals and work in fields they enjoy. Perhaps there are more women than men interested in medicine and public administration, in which case it’s probably better that more women than men work in those fields because that allows more people do something they most like to do. Maybe those facts can change with time, culture, or educational initiatives, one way or another. Our diversity initiatives would do best to reflect those realities, whatever they are, rather than fighting them, but then, that’s what James Damore said.

James A. Lindsay

James A. Lindsay is a thinker, not a philosopher, with a doctorate in math and background in physics. He is the author of four books, most recently Life in Light of Death. His essays have appeared in TIME, Scientific American, and The Philosophers’ Magazine. He thinks everybody is wrong about God.
Advertisements
If you enjoy our articles, be a part of our growth and help us produce more writing for you:
James A. Lindsay

James A. Lindsay is a thinker, not a philosopher, with a doctorate in math and background in physics. He is the author of four books, most recently Life in Light of Death. His essays have appeared in TIME, Scientific American, and The Philosophers’ Magazine. He thinks everybody is wrong about God.

6 thoughts on “A Primer on Statistics to Help Quell Your Outrage at the Google Memo

  1. Damore’s memo reminded me of Murray’s and Herrnstein’s Bell Curve. I wonder whether he thinks all blacks or just black women are suited for tech work?

    Well, if we are just going to make shit up about him, does anyone know where he was when that helicopter went down?

  2. I wonder whether he thinks all blacks or just black women are suited for tech work?

    Damore’s memo said that individuals should be treated as individuals, not as members of a group. I suspect he thinks that the shape of an individual’s skin is just as irrelevant as the color.

  3. Damore’s memo reminded me of Murray’s and Herrnstein’s Bell Curve. I wonder whether he thinks all blacks or just black women are suited for tech work?

  4. One of the inconsistencies on the Left is their approach to neurodiversity. If a condition is clinically significant they’ll defend the neuro-atypicals rights and celebrate their differences. Traits like autism, they’ll admit, are on a spectrum,

    But if those differences are sub-clinical they’ll deny their existence. If you are not neuro-atypical then they switch to the blank state model. Autistics prefer things to people because they have a developmental disorder but if men prefer things to people (albeit less so) that’s down to socialisation.

    It’s just a coincidence that there are more autistic men than women.

    This is actually more discriminatory because it posits two kinds of people; those born with innate preferences (autistics) and a majority born without (a blank state).

    That’s binary thinking. It’s incompatible with the concept of a spectrum on which everybody has a place, not just those with clinically significant conditions.

  5. Most of what Denmore was taking about preferences rather than IQ. He made the argument that men tend to be more interested in things rather than people and women tend to be interested in people rather than things.

    For this he was accused of advocating eugenics.

    Now, bits not controversial to say that people with autism spectrum disorders prefer things to people. Somehow, despite this being one of the diagnostic criteria, we seem to have avoided forced sterilisation and the gas chambers.

    Hell, Hans Asperger himself suggested that most geniuses have a touch of autism about them and he worked under the Nazis.

    There are a disproportionate number of aspies in IT. It’s not controversial to say that this reflects a preference for things rather than people.

    A disproportionate number of aspies are men. If you drew a distribution curve of any of the traits associated with Asperger’s (preference for things over people, obsessional interests, emotional reciprocity. etc) you’d end up with a graph similar to that of height distributions.

    There would be a considerable overlap between men and women but a difference in both mean and distribution.

    Asperger’s is basically where a constellation of all these traits occurs at the tail of the curve. But because the mean and deviation is different you are going to find more men than women at that tail end.

    Hence more male Aspies.

    Now I’m not remotely suggesting that all men in IT are aspies but the same reasoning that there are more male aspies than women: that there is a difference in both the averages and distribution of traits that are also used to assess autism – one of these traits being a preference for things rather than people.

    Other such traits would include obsessive interests, a preference for systems, etc.

    If you have all of these traits in excess you probably have autism but in moderation you could easily fall within the normal distribution for men but outside the normal distribution for women.

    (I hate to use the word normal here but that’s the term we are stuck with. It’s meant in the mathematical sense, not in the sense that a woman exhibiting more of these traits is ‘abnormal’)

    It’s funny that those denying this difference in aggregate are still using the nerd stereotype of IT techs as social inadequates don’t follow that logic to its conclusion.

LEAVE A REPLY