Average impact falls but the hits keep coming
Scientists are getting older.
Below is the share of employed US PhD scientists and engineers in three different age ranges: early career (under 40), mid-career (ages 40-55), and late career (ages 55-75). The figure covers the 26 years from 1993-2019.
Over this period, the share of mid-career scientists fell from about half to just under 40%. Most (but not all) of that decline has been offset by an increase in the share of late career scientists. And within the late career group, the share older than 65 has more than doubled to 27% over this time period.1
This trend is consistent across fields. Cui, Wu, and Evans (2022) look at more than one million scientists with fairly successful academic careers - they publish at least 10 articles over a span of at least 20 years. Cui and coauthors compute the share of these successful scientists who have been actively publishing for more than twenty years. Across all fields, it’s up significantly since 1980 (though, consistent with the previous figure, this trend may have peaked around 2015).
Alternatively, we can get some idea about the age of people doing active research by looking at the distribution of grants. At the NIH, the share of young principal investigators on R01 grants has dropped from a peak of 18% in 1983 to about 3% by 2010, while the share older than 65 has risen from almost nothing to above 6%.
This data ends in 2010, but the trend towards increasing age at receiving the first NIH grant has continued through 2020.
Is this a problem? What’s the relationship between age and innovation?
This is a big literature, but I’m going to focus on a few papers that use lots of data to get at the experience of more typical scientists and inventors, rather than the experience of the most elite (see Jones, Reedy and Weinberg 2014 for a good overview of an older literature that focuses primarily on elite scientists).
Yu et al. (2022) look at about 7mn biomedical research articles published between 1980 and 2009. Yu and coauthors do not know the age of the authors of the scientists who write these articles, but as a proxy they look at the time elapsed since their first publication. They then look at how various qualities of a scientific article change as a scientist gets older.
First up, data related to the citations ultimately received by a paper. On the left, we have the relationship between the career age of the first and last authors, and the total number of citations received by a paper.2 On the right, the same thing, but expressed as a measure of the diversity of the fields that cite a paper - the lower the number, the more the citations received are concentrated in a small number of fields. In each case, Yu and coauthors separately estimate the impact of the age of the first and last author.3 Note also, these are the effects that remain after controlling for a variety of other factors. In particular, the charts control for the typical qualities of a given author (i.e., they include author fixed effects). See the appendix for more on this issue. Also, they’re statistical estimates, so they have error bars, which I’ve omitted, but which do not change the overall trends.
The story is a straight-forward one. Pick any author at random, and on average the papers they publish earlier in their career, whether as first author or last author, will be more highly cited and cited by a more diverse group of fields, than a paper they publish later in their career.
In the figure below, Cui, Wu, and Evans (2022) provide some complementary data that goes beyond the life sciences, focusing their attention on scientists with successful careers lasting at least twenty years and once again proxying scientist age by the time elapsed since their first paper was published. They compute a measure of how disruptive a paper is, based on how often a paper is cited on it’s own, versus in conjunction with the papers it cites. The intuition of this disruption measure is that when a paper is disruptive, it renders older work obsolete and hence older work is no longer cited by future scientists working in the same area. By this measure, as scientists age their papers get less and less disruptive (also and separately, papers are becoming less and less disruptive over time, as discussed more here).4
Last up, we can even extend these findings to inventors. Kaltenberg, Jaffe, and Lachman (2021) study the correlation between age and various patent-related measures for a set of 1.5mn US inventors who were granted patents between 1976 and 2018. To estimate the age of inventors, Kaltenberg and coauthors scrape various directory websites that include birthday information for people with similar names as patentees, who also live in the same city as a patentee lists. They then compute the relationship between an inventor’s estimated age and and some version of each of the metrics discussed above. Once again, these results pertain to what remains after we adjust for other factors (including inventor fixed effects, discussed below).
On the left, we have total citations received by a patent. In the middle, a measure of the diversity of the technologies citing a patent (lower means citations come from a narrower set of technologies). And on the right, our measure of how disruptive a patent is, using the same measure as Cui, Wu, and Evans. It’s a by-now familiar story: as inventors age, the impact of their patented inventions (as measured by citations in various ways), goes down.
(The figures are for the patents of solo inventors, but the same trend is there for the average age of a team of inventors)
So in all three studies, we see similar effects: the typical paper/patent of an older scientist or inventor gets fewer citations and the citations it does get come from a smaller range of fields, and are increasingly likely to come bundled with citations to older work. And the magnitudes involved here are quite large. In Yu et al. (2022), the papers published when you begin a career earn 50-65% more citations than those published at the end of a career. The effects are even larger for the citations received by patentees.
This seems like pretty depressing news for active scientists and inventors: the average paper/patent gets less and less impactful with time. But in fact, this story is misleading, at least for scientists. Something quite surprising is going on under the surface.
Liu et al. (2018) study about 20,000 scientists and compute the probability, over a career, that for any given paper, their personal most highly cited paper lies in the future. The results of the previous section suggest this probability should fall pretty rapidly. At each career stage, your average citations are lower, and it would be natural to assume the best work you can produce will also tend to be lower impact, on average, than it was in earlier career stages.
But this is not what Liu and coauthors find! Instead, they find that any paper written, at any stage in your career, has about an equal probability of being your top cited paper!
The following figure illustrates their result. Each dot shows the probability that either the top cited paper (blue), second-most cited paper (green), or third-most cited paper (red) lies in the future, as you advance through your career (note it’s actually citations received within 10 years, and normalized by typical citations in your field/year). The vertical axis is this percent. The horizontal one is the stage in your career, measured as the fraction of all papers you will ever publish, that have been published so far.
This number can only go down, because that’s how time works (there can’t be a 50% chance your best work is in the future today, and a 60% chance it’s in the future tomorrow). But the figure shows it goes down in a very surprising way. Assuming each paper you publish has the same probability of being your career best, then when you are 25% of the way through your publishing career, there is a 25% chance your best work is behind you and a 75% chance it’s ahead of you. By the time you are 50% of the way through your publishing career, the probability the best is yet to come will have fallen to 50%. And so on. And that is precisely what the figure appears to show!
What’s going on? Well, Yu and coauthors show that the number of publications in a career is not constant. Through the first 20-25 years of a career, the number of publications a scientist attaches their name to seems to rise before falling sharply. Since the average is falling over this period, but the probability of a top cited paper is roughly constant, it must be that the variance is rising (the best get better, the worse get worse), in such a way that the net effect is a falling average.
And Yu and coauthors present evidence that is the case. In the figure below, we track the average number of citations that go to hit papers in two different ways. In dark blue, we simply have the additional citations to the top cited paper by career stage. Note, unlike average citations, it does not fall steadily to zero: instead, it actually rises (slightly) for the first 20 years!
In the light blue, Yu and coauthors do something interesting. They count how many papers the scientist published in their first 5 years; let’s say it is four papers. Then, for each of the next 5-year career stages, they find the four most highly cited papers (or however many the scientist managed to publish in the first 5 years) and plot the average number of extra citations received to this subset. This group does not fall steadily to zero either! Scientists put out just as many good papers through the middle of their career as they did when they were young; they just also put out a bunch of extra stuff that has low impact.
But there’s still some bad news.
First, Yu and coauthors still find a sharp fall off in both productivity and citations to top papers after 25 years of career experience. For a scientist who first published at the age of 25, that’s 50 years old. And as we saw at the beginning of this post, the share of scientists who fall into this “late career” demographic have been on the rise.
Second, it’s not clear if these trends apply at all to patented inventions. Kaltenberg, Jaffe, and Lachman find that among inventors, the annual number of patents peaks at a young age, around age 30, and then falls off steadily through the rest of the lifecycle.
More broadly, we only really have this data for the number of citations to papers; I am quite curious if something similar is going on with the disruption scores, or the diversity of impact. That would be quite interesting, because I think we also have a bunch of evidence that the nature of innovation changes as scientists age, and that might not show up in citation counts.
New articles and updates to existing articles are typically added every two weeks. To learn what’s new on New Things Under the Sun, subscribe to the newsletter.
Blau, David M., and Bruce A. Weinberg. 2017. Why the US science and engineering workforce is aging rapidly. PNAS 114(15) 3879-3884. https://doi.org/10.1073/pnas.1611748114
Cui, Haochuan, Lingfei Wu, and James A. Evans. 2022. Aging Scientists and Slowed Advance. arXiv 2202.04044. https://doi.org/10.48550/arXiv.2202.04044
Jones, Benjamin, E.J. Reedy, and Bruce A. Weinberg. 2014. Age and Scientific Genius. NBER Working Paper 19866. https://doi.org/10.3386/w19866
Yu, Huifeng, Gerald Marschke, Matthew B. Ross, Joseph Staudt and Bruce Weinberg. 2022. Publish or Perish: Selective Attrition as a Unifying Explanation for Patterns in Innovation over the Career. Journal of Human Resources 1219-10630R1. https://doi.org/10.3368/jhr.59.2.1219-10630R1
Wu, L., Wang, D. & Evans, J.A. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019). https://doi.org/10.1038/s41586-019-0941-9
Kaltenberg, Mary, Adam B. Jaffe, and Margie E. Lachman. 2021. Invention and the Life Course: Age Differences in Patenting. NBER Working Paper 28769. https://doi.org/10.3386/w28769
Liu, Lu, Yang Wang, Roberta Sinatra, C. Lee Giles, Chaoming Song, and Dan Wang. 2018. Hot streaks in artistic, cultural, and scientific careers. Nature 559: 396-399. https://doi.org/10.1038/s41586-018-0315-8
There is an important composition story here that the presentation above obscures: more talented scientists and inventors are more likely to stick around. That means, on average, the older is an active scientist/inventor, the more likely they are to be above average. This selection effect operates in opposition to the average aging effects documented above. If you pick a random scientist, the older they are, the more likely they are to be more talented (since they were good enough to remain in academia), but this talent is partially offset by the negative (average) effects of age. The net effect is ambiguous.
Here are the two figures by Yu et al. (2022) that I presented here, but no longer including these author fixed effects. Instead of comparing citations among the same scientist, at different points in their career, we are now comparing citations among older scientists in general to citations among younger scientists in general.
Now, we see the citations received by the average paper peak among mid-career scientists, and that the diversity of citing sources peaks in late career for last authored work! Note also the size of the effect of age is significantly smaller than it was in the version of this figure that compared only within an author.
Essentially, what’s happening here is that if you end up picking a paper with an older author, it’s increasingly likely you’ve picked a paper by a very talented scientist, since they had to be good enough to play the academic game a long time. Among younger scientists, that’s not the case: the typical paper by a young scientist includes many papers by scientists who will end up quitting academia. When you do not control for author-specific effects, you are no longer comparing two populations that are the same, except for their age. If you meet a random scientist who is 10-15 years into their career and pick one of their recent papers at random, it will probably receive more citations than a paper from a random scientist 5-10 years into their career. At the same time, it will probably have fewer citations than the older scientist received themselves on their papers, when they were 5-10 years into their career.
But once again, to the extent we care about the average citations received by a paper, selection effects seem to become overpowered by the penalties of aging for late career scientists, which is, again, precisely the group seeing the most rapid rise (though, on the other hand, the size of the effect isn’t enormous).