1 Introduction

Discrimination hurts its victims and, sometimes, its perpetrators (Becker 1957). For example, if an academic journal only publishes papers authored by men, its quality should decline relative to one that is gender-blind; if a male economist refuses to co-author with women, his papers ought to publish less well than the men who don’t.

For these reasons, discrimination is sometimes considered incompatible with competitive forces (see, e.g., Summers 2005). When markets are complete, the argument is roughly as follows: sufficient competition between unprejudiced journals should ensure female-authored papers are accepted at rates just equal to their marginal quality; sufficient competition between prejudiced and unprejudiced journals should ensure each is ranked according to the quality of the articles it publishes.1 As long as a journal’s ranking prices its articles’ quality, the quality of the papers it publishes should not vary by author gender nor should returns to co-authoring depend on a co-author’s sex.2

As we show in this paper, however, articles published in top economics journals authored by men are cited less than articles those same journals publish by women. Moreover, men’s citations rise when they co-author with women whereas women’s citations fall while they co-author with men, conditional on acceptance.

These results are based on an analysis of gender differences in citations in over 11,000 full-length articles published in a “top-five” economics journal—i.e., American Economic Review (AER), Econometrica (ECA), Journal of Political Economy (JPE), Quarterly Journal of Economics (QJE) and Review of Economic Studies (REStud). We follow previous research and use the inverse hyperbolic sine (asinh) of citations as our dependent variable (Card and DellaVigna 2020; Card et al. 2020). We stress, however, that results and conclusions are similar if raw citations or alternative transformations are used instead (see Appendices B and C).

Our first analysis suggests female-authored papers receive 22–24 log points more citations than male-authored papers. Differences in citation patterns across field coupled with higher female concentration in certain areas potentially explain, at most, half of the gap. Assuming quality positively associates with citations and the latter are not biased in favour of women (conditional on the former), these results suggest female-authored papers published in top economics journals are higher quality than male-authored papers.

Higher quality female-authored papers could be consistent with gender-neutral acceptance standards if women’s papers are accepted more often or the variance in their quality is greater (see Theorem 3.1). Neither appears to be the case. Variance in quality is persistently lower in female-authored papers; recent evidence from a set of journals that partially overlaps with our own suggests men’s and women’s manuscripts are accepted at roughly equivalent rates (Card et al. 2020).

We also consider whether consistent same- or opposite-sex co-author complementarities meaningfully contribute to gender differences in quality, conditional on acceptance. For example, men and women may produce higher quality work when collaborating with one another; conversely, everybody could work better with members of their own sex.

Our evidence does not support either hypothesis. After accounting for author-specific fixed effects, we find men’s citations increase 10–12 log points when they co-author with an equal share of women (compared to papers those same men co-author entirely with other men). The returns to women of co-authoring with an equal share of men, however, are -12 to -15 log points. These findings suggest women disproportionately contribute to the quality of co-authored work, conditional on acceptance, and are consistent with women facing tougher standards in peer review.

Moreover, the coefficients on co-author gender do not meaningfully change after accounting for field. This points to a relationship between citations and gender that is independent of field, conditional on author. Considered alongside the fall in coefficient value from our first analysis, it may also reveal an underlying association between field and author-specific unobservables that will partially bias estimates of gender differences in citations when controlling for the former but not the latter.

As a final exercise, we restrict our sample to senior male economists with at least two top-five papers co-authored with a single junior author of each sex. In addition to accounting for potential unobserved contributions from same-sex co-authors, these sample restrictions create a treatment group—senior male authors co-authoring with exactly one junior woman—that very closely resembles the counterfactual group—those very same seniors co-authoring with exactly one junior man.

Again, we find a senior man’s work is higher quality when it is co-authored with a woman. Citations increase 60–70 log points—and 140 log points after accounting for field—when senior male economists co-author with junior women as opposed to junior men.

Combined, our evidence suggests journals subject female authors to higher standards and, as a result, their articles are better quality, conditional on acceptance. We emphasise, however, that these conclusions rely on the following (strong) assumptions: (i) citations are not biased in women’s favour, conditional on quality; (ii) author-specific heterogeneity is fixed over time (author-level analyses, only); (iii) quality is normally (but not necessarily identically) distributed in male- and female-authored submissions; and (iv) conditional on controls, male- and female-authored papers are accepted at similar rates.

Assuming (i)–(iv) are satisfied, non-top-five economics journals could have published higher quality content than top-five economics journals simply by accepting more female-authored papers. The fact that journal rankings have not adjusted to reflect this suggests the market for academic research remains incomplete.

Journals function as price mechanisms—i.e., the journals in which articles are published serve as nominal currency for their value. If women could hedge (without friction) against every possible publication outcome in every possible state of the world, then biased acceptance decisions at one journal could simply be “undone” by a costless change in one’s submission and publication strategy the previous date—e.g., women could publish their higher quality papers in currently lower-tiered journals, confident that their actions would lead to an appropriate relative change in journal rankings the very next period.

When competition isn’t perfect, however, discrimination interacts with one or more market frictions to prevent those who discriminate from fully internalising its costs. Consequently, its victims will have to partially bear them. For example, imperfect information about journal rankings may mean tenure and promotion committees’ expectations are slow to adjust to the lower quality of journals that reject too many women.3 As a result, women (and the men they co-author with) are tenured and promoted at lower rates than they otherwise would be if markets were complete. To the extent that grant committees similarly rely on applicants’ past publication histories to choose between projects, women will also have a harder time funding future work.

Moreover, discrimination undoubtedly distorts authors’ decisions in ways that can further misallocate available resources. Indeed, our own evidence implies male and female economists are better off collaborating with men, all else equal. This incentivises authors of both sexes to forgo higher quality co-authoring opportunities with women in order to partner with men (see also Knobloch-Westerwick, Glynn, and Huge 2013).

This paper makes four contributions. First, we join a large literature investigating gender differences in citations in economics (see, e.g., Ferber 1988; Laband 1987; Smart and Waldfogel 1996; Ginther and Kahn 2004; Hamermesh 2018; Grossbard, Yilmazer, and Zhang 2018; Card et al. 2020). We add to this research by studying the comprehensive set of journals at the very top of the publishing hierarchy and examining gender-specific contributions to quality in mixed-sex co-authoring relationships, conditional on acceptance.

Second, we also contribute to a substantial body of research suggesting women are, in many situations, subjected to tougher standards and/or evaluated differently than men (see, e.g., Foschi 1996; Moss-Racusin et al. 2012; Reuben, Sapienza, and Zingales 2014; Krawczyk and Smyk 2016; Sarsons et al. 2019). Most relevant to our work, Hengel (2019) finds female-authored papers in top-four economics journals are held to higher writing standards in academic peer review. As a result, their manuscripts are subject to greater scrutiny, spend longer under review and women, in turn, respond by conforming to those standards. Card et al. (2020) finds female-authored papers are higher quality conditional on referee recommendations using submissions data from a partially overlapping set of journals (Journal of the European Economic Association, Review of Economics and Statistics, QJE and REStud).

Third, although market mechanisms undoubtedly alleviate discrimination’s downstream effect, our results highlight that they will not, in general, fully absorb them. For example, the fall in women’s citations when they co-author with men suggests adding male co-authors can mitigate higher acceptance standards; men also experience a rise in citations when they co-author with women, however, so co-authorship alone probably cannot eliminate them. And when combined with recent evidence of the “Matilda effect” in tenure decisions—as Sarsons et al. (2019) shows, tenure committees discount women’s contributions to mixed-sex co-authored work—co-authoring with men may bring other consequences, as well.

Finally, this paper builds on a broader literature studying editorial patterns (Card and DellaVigna 2013; Casnici et al. 2017; Clain and Leppel 2018; Ellison 2002), bias in editorial decisions (Abrevaya and Hamermesh 2012; Bransch and Kvansnicka 2017; Card and DellaVigna 2020; Hospido and Sanz 2019) and female academics’ lagging productivity and under-representation (Bayer and Rouse 2016; Ductor, Goyal, and Prummer 2018; Ginther and Kahn 2004; Teele and Thelen 2017; Lundberg and Stearns 2019; Auriol, Friebel, and Wilhelm 2019; Chari and Goldsmith-Pinkham 2017; Heckman and Moktan 2019). We also join a wider debate about whether women are considered equal partners in research and given enough credit for their contributions (see, e.g., Ferber 1986; Dion, Sumner, and Mitchell 2018; Sarsons et al. 2019).

Our paper proceeds in the following order. Section 2 describes the data. Section 3 assesses gender differences in citations and quality. Section 4 investigates the returns to co-authoring with the opposite sex. Section 5 concludes.

2 Data and empirical setting

Our data include 11,336 full-length, English-language articles published between 1950–2015 in the AER, ECA, JPE, QJE and REStud. We define “full-length” as any article published with an abstract. Articles from Papers & Proceedings issues of AER, errata and corrigenda are excluded.

Each of the 7,574 unique authors in our dataset was manually assigned a gender based on (i) obviously gendered given names (e.g., “James” or “Brenda”); (ii) photographs on personal or faculty websites; (iii) personal pronouns used in text written about the individual; and (iv) by contacting the author himself or people and institutions connected to him.

As we highlight in Section 2.2, exclusively female-authored papers are only a very small proportion of articles published in top economics journals. We therefore define the gender of a paper by its fraction of female authors. (See Hengel (2019) for a theoretical justification of this indicator.) For robustness, we also replicate relevant analyses using a categorical variable to account for seven different gender categories and by comparing entirely male-authored papers to papers with: (a) a senior female author; (b) at least one female author; (c) a weak majority of female authors; and (d) a solo female author. Results are shown in Appendix D.

Citation data were obtained from Web of Science (2018), a comprehensive database of all social science research published since 1900. Counts correspond to the number of published papers in the Web of Science database that cite a given article and include self-citations to later work. Citations for AER, ECA, JPE and QJE were first collected in August 2017 and updated in January 2018; citations for REStud were collected in October 2018.

2.1 Issues when analysing gender citation gaps

Analysing citation gaps raises several issues: (i) younger articles have had less time to accrue citations; (ii) older articles are disproportionately male; (iii) mean citation counts may be distorted by a small number of superstar economists; (iv) superstar economists are also disproportionately male. (i) and (iii) skew the distribution of citations; (ii) and (iv) distort gender differences if they correlate with unobserved factors that generate citations—e.g., winning a prestigious award—or produce self-reinforcing loops that create citations all on their own (Merton 1986, 1988; Zuckerman 1977).4

Additionally, men disproportionately cluster at the very top and bottom of the distribution of citations but raw counts are censored below at zero and unbounded from above (Figure 2.1, Graph (A)). This generates a non-linear mapping from quality onto citations that depends on the former’s variance. Thus even in the absence of (i)–(iv), average citations to male-authored papers likely place too much weight on high-citation papers and not enough weight on low-citation papers compared to the average for female-authored papers.5

Distribution of citations

Note. Left-hand graph displays the fraction of authors with a top-five paper that was cited 0–15 times, 16–50 times, 51–100 times, etc. Right-hand graph plots the histogram of transformed citations (asinh).

Figure 2.1: Distribution of citations

We principally account for these issues by controlling for journal-year fixed effects and transforming raw counts with the inverse hyperbolic sine function (asinh). This reduces the impact of outlier observations and results in a far more symmetric distribution (Figure 2.1, Graph (B)). We emphasise, however, that the conclusions we draw do not depend on the transformation. We find similar results using a quantile regression model (Appendix B) and the log of 1 plus citations or raw counts themselves as the dependent variable (see Appendix C).

A related concern is that the citations a paper accumulates aren’t fixed in time. As a result, they could be influenced by the future success or failure of a paper’s authors—i.e., even among non-superstar economists, a stronger publishing record later on probably drives citations to earlier work, all else equal (for evidence see, e.g., Bjarnason and Sigfusdottir 2002). Female economists, however, have weaker publishing histories: men’s mean number of top-five papers is 2.6; the corresponding figure for women is only 1.8.

To account for this, we additionally control for how prominent an author is at about the time citations were collected (\(\text{max. } T\)). \(\text{Max. } T\) is equal to the total number of top-five articles for the most prolific co-author as of the end of 2015. Because ex post success may itself be endogenous to citations, we also show results without \(\text{max. }T\).

Finally, several studies show citation counts correlate with sub-field and author prestige at the time of publication, conditional on quality (see, e.g., Bornmann et al. 2012; Wang 2014). Additionally, more co-authors can present a single paper more widely and self-cite it more frequently.6 We therefore account for the following in most of our analyses: primary and tertiary JEL fixed effects,7 number of co-authors (\(N\)) and author seniority at the time of publication (\(\text{max. } t\))—i.e., the most prolific co-author’s total number of top-five articles at the time the paper was published.

2.2 The representation of women in top-five economics journals

Women are under-represented in top-five economics journals. As Graph (A) in Figure 2.2 illustrates, the situation has improved little with time: women make up only 11 percent of all authors published since 1990, 12 percent since 2000 and 14 percent since 2010. Between 1986–2015, there has been zero growth in the number of exclusively female-authored papers; almost no growth in the number of majority female-authored papers; and no meaningful change in the number of mixed-gendered papers with a senior female co-author. The only tepid growth that has occurred, is largely—if not entirely—due to an increase in the number of articles by senior men co-authoring with a weak minority of junior women.