An example of successful innovation by distributed teams: academia
·
It’s long been assumed that the best sorts of innovation happen when smart people work in an environment where spontaneous face-to-face interaction is the norm. Importantly, if that’s true, it implies the widespread transition to more remote work - where spontaneous face-to-face interaction is not possible - poses a threat to innovation. I’ve written about theseideasalotelsewhere, but this week, I want to look at a case study for a sector that:
engages in frontier knowledge work
has strong incentives to adopt practices that produce better outcomes
has been well studied
has increasingly moved to a model of remote collaboration
I am talking, of course, about academia.
Remote Collaboration in Academia
Academic research is a sector where knowledge workers try to innovate - the whole game is trying to push the knowledge frontier outward. Whether or not the system could work better, it certainly does seem to work, generating new and useful knowledge pretty much every day. It’s also a system that’s highly competitive, where thousands of individuals compete with each other for jobs and space in journals.
And yet, despite strong incentives to use any possible edge to generate new and better research, academics are increasingly forgoing the option to work with their local colleagues.
Frey and Presidente (2022) and Lin, Frey, and Wu (2022) each pull data on millions of papers from the Microsoft Academic Graph and compute the extent of remote collaboration from the affiliations of listed authors. The figures below present some summary statistics. At left (from Frey and Presidente 2022), we see the share of papers by authors who list different institutions rose from under 10% in 1960 to nearly 50% by 2020. At right (from Lin, Frey, and Wu 2022), we can see the average distance between teams of collaborators rose from under 200km to nearly 1,000km over the same time period. Other papers document similar trends.1
All told, the share of papers produced by academic teams primarily engaged in remote collaboration - rather than primarily local face-to-face interaction - has been on the rise for decades, and now accounts for nearly half of all papers.
Agrawal, McHale, and Oettl (2017) looks at what happens to the research productivity of an evolutionary biology department when they successfully recruit a star researcher to join the department (“stars” are defined to be researchers whose citation-weighted publication count is in the top 10%). In one exercise, they take the set of academics working in the department before it gets a star hire, and divide them into two groups: those who have previously cited the star’s work (“related” scientists) and those who haven’t. The productivity (annual citation-weighted publications) of scientists working on related stuff goes up 96% in the years after a star is hired!
Now, a potential objection to this paper is that star researchers don’t just get randomly allocated to different departments. It could be that departments that are already “on-the-up” are the ones that attract stars. Maybe, for example, a department has just secured a big endowment that lets them go on a hiring binge and provide better resources to everyone in the departments. In that case, we might falsely attribute higher research productivity that is actually due to more resources to the appearance of a star researcher. But if that was the case, we would expect to see research productivity for the department to rise for everyone around when a star is hired - and we don’t. Instead, scientists unrelated to the star see no impact from the star’s arrival. (Agrawal, Oettl, and McHale also try to answer this concern in a few other ways, but don’t find anything that alters their core conclusion).
Frey and Presidente (2022) widens our frame to most of academia2 and look at what happens to various characteristics of the research performed by a team of colleagues when they are together (i.e., all list the same institutional affiliation) versus when they are apart (i.e., list different affiliations). They find over 1960-2020, a team produces 2.7% fewer papers per year when it is distributed, as compared to when it is colocated. The papers they produce when separated also receive 9.5% fewer citations, as compared to when they are together. And using a measure of how disruptive a paper is (based on whether people who cite it keep citing older literature or not), they find papers by remote teams are less disruptive than the papers of colocated teams (with a very important caveat that we’ll come back to).
Like Agrawal, McHale, and Oettl (2017), they need to worry a bit about what else might be going on when a team changes from working together to working apart, or vice versa. Again - people don’t move for random reasons. Teams that experience a transition are likely to be teams that are older, simply because it takes time to move. They’re also teams where at least someone on the team changed jobs. Moreover, moving is a disruptive process, and it could be that research gets disrupted when a colleague moves away, but only temporarily. Frey and Presidente try to control for these possible confounders, but they continue to find a negative impact of being remote versus being together, at least in terms of the disruptiveness of a paper (which is their primary focus).
So both these results are a bit puzzling, given the trend towards more distributed academic teams. It also contradicts a few other studies. Let’s look at those studies and come back to these two.
Dubois, Rochet, and Schlenker (2014) looks at the same time period for the set of all mathematicians (in the world) who published at least two articles in a mainstream math journal between 1984 and 2006. Rather than look at the impact of an individual researcher on the productivity of a department though, it looks at the impact of a department on the productivity of an individual. By keeping track of the moves of individual academics, they can see how the productivity of researchers changes when they move to top universities. If being able to have frequent face-to-face interactions with other great researchers is really important for producing good research, then we ought to see the research productivity of mathematicians jump up when they move to a department full of great researchers.
While it is true that top departments have more productive faculty, it turns out this is just because they hire more productive faculty. Once you take into account the productivity of a researcher (i.e., from before they get hired to a top department), the authors don’t find any increase in productivity when these people move to top departments. It seems that the ability to have lots of face-to-face interactions with other top mathematicians is not doing much to raise their productivity (which was already high).
Ultimately, these are both observational studies, and so we might be a bit worried that something funny is going on to deliver these results. In an ideal experiment, we would randomly shuffle around researchers, looking at the impact on departments that randomly receive (or lose) a researcher to those that do not. We can’t do that, but two other papers identify some (grim) natural experiments where faculty in some department suddenly lose the opportunity to have frequent face-to-face interactions with a top researcher. In all cases, there doesn’t seem to be much value to it.
Waldinger (2012) looks at German physics, chemistry, and mathematics departments over 1925-1938. In 1933, there was an awful event that led to some departments losing colleagues and other departments not, for reasons quite unlikely to be correlated with factors driving academic productivity: the Nazi dismissal of Jewish and politically “undesirable” scientists. Like Agrawal, Oettl, and McHale, Waldinger looks at the research productivity of the faculty who were left behind. And, surprisingly, he finds no impact. The subsequent research trajectory of faculty in departments that lost local colleagues were no different than departments that didn’t lose anyone. (This isn’t to say there was no effect; as he documented in Waldinger 2016 - departments that dismissed faculty saw total output drop, and were never able to recover by hiring replacement faculty with the same research output).
Azoulay, Graff Zivin, and Wang (2010) are also interested in the impact of losing a colleague: they use sudden and unexpected deaths as a plausibly random shock to a department. Azoulay and coauthors identify 10,349 elite life scientists, as well as their coauthors. They then further identify a subsample of 112 eminent life scientists who died suddenly and unexpectedly between 1979 and 2003 before the age of 67, and who were still active researchers at the time of their death. It is the coauthors of these eminent scientists that they study. They match each scientist who experienced a sudden and unexpected death of an eminent coauthor to a similar scientist working with another eminent coauthor who did not die.
They find the unexpected death of a colleague reduces output by 5-8% relative to those who did not. Of this group, about 12% of coauthors were colocated with the deceased at the time of their death. But when Azoulay, Graff Zivin, and Wang look to see if this group is more adversely affected by the death than distant coauthors, they find just a small effect (in the opposite direction) that can’t be statistically distinguished from zero. Losing a coauthor hurts; but in terms of it’s impact on measurable research outputs (setting aside the real human cost), it doesn’t look like coauthors able to engage in frequent face-to-face interaction were any more affected than those who couldn’t.
So we have five studies now. One finds that evolutionary biologists benefit (a lot) when a talented colleague working on similar stuff joins the department. Another finds that when a team collaborates at a distance, the quality of research suffers as compared to when they are all together. The other three find local access to a colleague, however, doesn’t seem to have much impact. And Agrawal, McHale, and Oettle actually document that long-distance collaboration is becoming the norm in evolutionary biology, precisely the field where another study documents strong benefits to a local star researcher. How can we square these results?
I think there are two things going on. First, (as I’ve argued elsewhere) I suspect proximity might be quite important for forming working relationships, but less important for maintaining them. Second, the importance of local colleagues is declining over time, and most of the results on the benefits of proximity are driven by data from an era when the effects of locality were stronger than they are today. Let’s start with the former.
Getting to know you
Suppose local face-to-face interactions are important for forming new relationships but less important for doing productive work once a relationship has been formed. In that case, we would expect hiring a new star to boost the output of local researchers working on related topics, because they could form a new collaborative partnership. And that’s basically what Agrawal, Oettl, and McHale (2017) find. Most of the increased output of evolutionary biologists when they hire a new star research seems to stem from publications they coauthored with the star.
Conversely, if the opportunity to collaborate disappears, everyone who would normally collaborate with them - near and far - would be equally adversely affected. That’s essentially what Waldinger (2012) and Azoulay, Graff Zivin, and Wang (2010) find. Waldinger notes that coauthorship across departments was already pretty common in Germany before 1933 because Germany had a well integrated national research community with frequent national conferences and mail correspondence about scientific research. Azoulay, Graff Zivin, and Wang’s study focuses on people who were already coauthors with the deceased - definitionally a group that has likely already cleared the threshold of “forming a relationship” - and no longer needs proximity to work together productively. So when the opportunity to collaborate disappears, due to one tragedy or another, they’re equally negatively impacted, whether they are in the department or not.
We’ve got some complementary evidence that colocation is important for forming academic working relationships. In Frey and Presidente (2022)’s data, more than 70% of teams who change their mode of collaboration go from being colocated to distributed, rather than from distributed to colocated. That suggests most teams initially form among people who are physically close to each other.
More direct evidence comes from a figure I’ve reproduced elsewhere, by Freeman, Ganguli, and Murciano-Goroff (2015). They survey teams of collaborators in three different fields,3 to ask how the academic collaborators first met:
Look at how non-colocated and international collaborators first met: 90% of the time, it was during some kind of face-to-face interaction.
Change over time
In Agrawal, McHale, and Oettl (2017), most of the new hires of star researchers that they observe occur prior to 1995. That might be important, because we have a number of papers that suggest the benefits of local face-to-face interactions where stronger prior to the year 2000.
Frey and Presidente (2022) document this change over time quite directly. Focusing on the disruptiveness of research, they find the penalty for a remote team declines over time and by the year 2000, we cannot reject the null that there is no penalty for a remote team at all. Indeed, by the year 2015, remote teams do more disruptive work than colocated ones (see this post for more discussion on the topic of remote teams and disruptive/breakthrough innovation)!
Other papers also document declining importance of colocation. The most direct test of this is Kim, Morse, and Zingales (2009), which examines the research productivity of academics in economics and finance over 1970-2000. Like Dubois, Rochet, and Schlenker (2014), they’re going to try and identify the productivity bump a researcher gets from moving to a top department by tracking the moves of individual academics over their careers. In this case, research productivity is measured in terms of how many articles an economist/finance professor writes per year, adjusted for the quality of journal. They actually do find that moving to a top department helps researchers be more productive - in the 1970s. But the productivity bonus researchers got for moving to a top 25 university dropped in the 1980s, and had disappeared by the 1990s.
Two other recent papers zero in on the flow of knowledge across space, as measured by citations. Head, Li, and Minondo (2019) looks at the probability that there is a citation between two different mathematics papers. In the figure below, they plot how the elasticity of distance changes over time - roughly speaking, the more negative, the less likely a citation between math papers by authors who are far apart.
In the left figure, in orange, we see the distance penalty has dramatically shrunk between 1990 and 2010. That’s the main finding of interest for today - being close matters less and less, in terms of the probability two researchers know about each other’s work. (See this post for a bit more discussion of this paper). In the right figure, the authors break this out for citations between US papers (in orange) and citations between US and non-US countries. Surprisingly (to me), there has actually never been a strong penalty for distant papers in the USA, at least since the 1990s. But the probability a paper from abroad is cited used to be substantially affected by distance and no longer is.
Hellmanzik and Kuld (2020) perform a similar exercise for economics, but focused entirely on international citations. They begin with a gravity model, which is traditionally used to statistically model the flow of exports and imports between countries, but then adapt it to model the flow of citations between economics papers written by authors in different countries. After controlling for a variety of factors, they find papers in 1979-1988 were nearly 6 times as likely to cite papers from their own country as compared to international ones, but by 2009-2016 they were just 1.4 times as likely to cite papers from their own country. Measures of the “distance” penalty, wherein countries that are farther away are even less likely to be cited also declined over the same time period.
Hellmanzik and Kuld also introduce a new measure that helps account for why some countries are more likely to cite papers from each other (again, after adjusting for lots of other potentially important stuff like language): the number of hyperlinks from one country to another. As you might expect, they find countries that are more interconnected by the internet are more likely to cite each other’s work, and that this effect has grown stronger over time.4 Frey and Presidente (2022) obtain some additional confirmatory evidence: among papers authored by teams belonging to OECD countries, they find the penalty for distance is weaker in countries and years when broadband internet subscriptions per capita are higher.
Taken together, I read this as providing a different but complementary line of evidence that local interaction is not as important as it used to be for doing frontier knowledge creation.
The Upsides of Remote Collaboration
So I think proximity was (and perhaps still is) important for forming collaborative working relationships, but these relationships remain productive even when academics are far away from each other. At least, this is increasingly the case.
But at the same time, I think there’s another factor that increasingly pushes academics to collaborate remotely. That is the ever-rising set of knowledge needed to push the frontier, which academics are responding to by drilling into ever narrower specialties and forming teams of specialists to tackle research projects (see this for more on this idea). When academics specialize more and more, it becomes ever less likely that the specialty you need to complete a project happens to reside in the same department.
We’ve got a few pieces of evidence in support of this view.
First, Freeman, Ganguli, and Murciano-Goroff (2015) explicitly ask the corresponding author on a bunch of publications what each coauthor brings to the table, so to speak. Specifically they ask “Did any of the team members working on this article (including yourself ) have access to one of the following resources that the other team members did NOT have, which made it important for you to all work together on this topic?”
Across every possible answer, colocated authors were less likely to bring a unique capability to the collaboration, as compared to non-colocated authors. For example, among non-colocated teams, 80% of the respondents said a team member brought unique knowledge, expertise or capabilities” to the collaboration, compared to 70% among colocated teams. Similarly, among 57% of non-colocated teams, someone brought unique data, materials or components to the collaboration, compared to 43% among colocated teams.
Azoulay, Graff Zivin, and Wang (2010) provide some similar evidence. They perform a series of exercises to try and sniff out what kinds of factors exacerbate the decrease in research productivity that comes when an eminent coauthor passes away. For example, they look to see if the negative impact of a coauthor’s death is stronger when the coauthor has plausibly better access to NIH grant funding. It isn’t. They look to see if it’s stronger when the coauthor has better potential to network in the field. Again, it isn’t.
Lastly, they look to see if the negative effect of a coauthors death is stronger when the coauthor works on more similar topics. This time, they find a robust correlation. That again suggests these collaborative partnerships are formed as a way to acquire the knowledge necessary to tackle hard problems, rather than as a way to share grant funding or curry favor in the scientific hierarchy.
Agrawal, McHale, and Oettl (2017) also get some results that I think also underscore how specialized science is becoming. Recall this is the paper that found evolutionary biologists benefit a lot if they hire a new star researcher, but only if they’ve previously cited their work. It turns out, this isn’t very common: just 9% of the faculty in a typical department are related to the star in this way. The other 91% of scientists in the field of evolutionary biology apparently work on subfields too dissimilar to benefit from the opportunity to collaborate with a star in evolutionary biology.
Agrawal, Oettl, and McHale (2015) also find some additional suggestive evidence consistent with this story. They find that, over time, stars have been producing a growing share of all citation-weighted work. They argue this is because stars build larger pools of potential collaborators (for example, by training more doctoral students) which becomes increasingly important as it gets more important to find partners with just the right specialty. A bigger pool to draw on also lets stars be more selective. Their paper provides evidence that stars do indeed produce papers with a larger set of unique coauthors, and that they have grown more selective over time.
Two cheers for dispersed teams!
In sum then, my story is basically this.
it’s not actually that hard to collaborate productively at a distance in academia, at least once you’ve gotten to know someone.
innovation requires ever more collaboration among specialists as knowledge accumulates.
Over time, falling travel and communication costs have increasingly favored building those teams by turning to remote colleagues with the right specialization.
I think these trends are a harbinger for things to come for the non-academic world (to some extent it’s already happening). Innovation by distributed teams will probably become more and more important as access to just the right specialty becomes ever more important and falling communication costs make it ever easier to do this well. To the extent innovation via dispersed teams has been shown to work pretty well in academia, that’s a reason not to be too concerned about it in other domains.
But there are caveats.
First, the nature of academic work might be particularly well-suited to innovation by distributed teams, as compared to other forms non-academic innovation. Projects are discrete, and finite, and many forms of academic projects have a modular character - one person can do the experiment, another can analyze the data, and so on. They also ultimately hinge on the creation of a digital artifact - the paper - rather than a physical prototype (although in many cases there is a lot of work that happens in the physical world on the way to a paper). Still, it might be that innovation in hardware, for example, doesn’t work nearly so well with a distributed team.
Second, while I think it’s quite feasible to productively work together at a distance, I suspect that academia still relies heavily on face-to-face interaction for one crucial step in the innovation process: meeting new people.5 If we’re thinking about applying the lessons of academia to non-academic examples of innovation by distributed teams, much hinges on whether colleagues at fully remote organizations get to know each other as well as they would if they were all in the office. As best as I can tell, this is certainly possible, but some deliberate planning is often necessary.
Besides the departmental work-unit, academia has evolved a rich set of mechanisms to circulate researchers and give them opportunities to form relationships with people who are unlikely to be colocated colleagues for the majority of a career. On the longer side, these include training - often in one place for a PhD and another for a postdoc. But they also include the seminar circuit and frequent academic conferences. Collectively, these temporary nexuses account for another 40-50% of non-colocated collaborative relationships in Freeman, Ganguli, and Murciano-Goroff’s survey. Non-academic sectors that hope to be as effective as academia at innovation with distributed teams may need similar distant networking opportunities, whether those take the form of apprenticeships, conferences, online social networks, etc.
New articles and updates to existing articles are typically added to this site every two weeks. To learn what’s new on New Things Under the Sun, subscribe to the newsletter.
Frey, Carl Benedikt, and Giorgio Presidente. 2022. Disrupting Science. Working Paper.
Lin, Yiling, Carl Benedikt Frey, and Lingfei Wu. 2022. Remote collaboration fuses fewer breakthrough ideas. arXiv:2206.01878. https://doi.org/10.48550/arXiv.2206.01878
Agrawal, Ajay, John McHale, and Alexander Oettl. 2015. Collaboration, Stars, and the Changing Organization of Science: Evidence from Evolutionary Biology. In The Changing Frontier: Rethinking Science and Innovation Policy, eds. Adam B. Jaffe and Benjamin F. Jones, pgs. 75-102. http://www.nber.org/chapters/c13038
Freeman, Richard B., Ina Ganguli, Raviv Murciano-Goroff. 2015. Why and Wherefore of Increased Scientific Collaboration. In The Changing Frontier: Rethinking Science and Innovation Policy, eds. Adam B. Jaffe and Benjamin F. Jones, pgs. 17-48. http://www.nber.org/chapters/c13040
Clancy, Matthew. 2020. The Case for Remote Work. The Entrepreneurs Network Briefing Paper.
Agrawal, Ajay, John McHale, and Alexander Oettl. 2017. How stars matter: Recruiting and peer effects in evolutionary biology. Research Policy 46(4): 853-867. https://doi.org/10.1016/j.respol.2017.02.007
Dubois, Pierre, Jean-Charles Rochet, and Jean-Marc Schlenker. 2014. Productivity and mobility in academic research: evidence from mathematicians. Scientometrics 98: 1669-1701. https://doi.org/10.1007/s11192-013-1112-7
Waldinger, Fabian. 2012. Peer Effects in Science: Evidence from the Dismissal of Scientists in Nazi Germany. The Review of Economic Studies 79(2): 838-861. https://doi.org/10.1093/restud/rdr029
Waldinger, Fabian. 2016. Bombs, Brains, and Science: The Role of Human and Physical Capital for the Creation of Scientific Knowledge. The Review of Economics and Statistics 98(5): 811-831. https://doi.org/10.1162/REST_a_00565
Azoulay, Pierre, Joshua S. Graff Zivin, and Jialan Wang. 2010. Superstar Extinction. The Quarterly Journal of Economics 125(2): 549-589. https://doi.org/10.1162/qjec.2010.125.2.549
Kim, E. Han, Adair Morse, and Luigi Zingales. 2009. Are elite universities losing their competitive edge? Journal of Financial Economics 93(3): 353-381. https://doi.org/10.1016/j.jfineco.2008.09.007
Head, Keith, Yao Amber Li, and Asier Minondo. 2019. Geography, Ties, and Knowledge Flows: Evidence from Citations in Mathematics. The Review of Economics and Statistics 101(4): 713-727. https://doi.org/10.1162/rest_a_00771
Hellmanzik, Christiane, and Lukas Kuld. 2021. No place like home: geography and culture in the dissemination of economic research articles. Empirical Economics 61: 201-229. https://doi.org/10.1007/s00181-020-01860-0
I wonder if there is strong evidence for this. I.e., what would be the counterfactual of not having elite universities or institutions like Bell Labs?
Would economic growth be severely slowed, due to the importance of small groups of exceptional researchers? Or is there a larger share of researchers who could come up with some idea, clusters of exceptional individuals merely being first? Lmk if you know of sources/articles on this, I’m writing a blog post that rests on the former assumption.