Chapter 7 Discomforting results about invasion hypotheses
Jonathan M. Jeschke
Leibniz Institute of Freshwater Ecology and Inland Fisheries (IGB), Berlin, Germany.
Institute of Biology, Freie Universität Berlin, Berlin, Germany.
Berlin-Brandenburg Institute of Advanced Biodiversity Research (BBIB), Berlin, Germany.
The paper I chose for my back story was the key outcome of a grant I held from the German Research Foundation (Deutsche Forschungsgemeinschaft DFG) from 2010 to 2013. The grant paid the postdoctoral position I had at the time in Munich, first at Ludwig-Maximilians-Universität München (LMU), Munich’s full university where I was based at the Planegg-Martinsried campus in the southwest of Munich, then at the Technical University of Munich (TUM) where I was based at the Restoration Ecology group located at the Freising-Weihenstephan campus in the northeast of Munich. Earlier in my career, I studied biology at LMU and also completed my doctorate there, however the ecology groups were located in Munich’s city centre at that time (before the valuable property was sold, with the university buildings being replaced by expensive apartments and a five-star hotel).
After my dissertation, I participated in a project at LMU’s Biology Education group and then moved to the USA to do a postdoc with Dave Strayer at the Institute of Ecosystem Studies in Millbrook, NY, USA, which was later renamed to Cary Institute of Ecosystem Studies. Although I was initially a bit skeptical about my life in the USA, particularly as I moved there alone in 2003 and left all my family and friends in Europe (I had not yet met my wife at that time), these two postdoctoral years were truly life-changing for me, both scientifically and personally. I was amazed how quickly I was integrated in the social life and how much I learned from Dave Strayer and other great scientists there. I was very happy to get to know Sujay Kaushal, Lorena Gómez Aparicio and others who were postdocs at the same time in Millbrook, and who became life-long friends. Having had such a great time in Millbrook during my postdoc, I became a Visiting Scientist at the Cary Institute for about a decade (2006-2017), during which I usually visited once or twice a year.
It was also in this time period that I worked with my co-authors on the focal paper of this back story: Jeschke et al. (2012b)
A related paper is Jeschke et al. (2012a), and both publications resulted from a sub-project we called the ‘Synthesis Project on Invasive Plants and Animals’ (SPIPA). As Jeschke et al. (2012a) was easier to publish, I focus on Jeschke et al. (2012b) here. Some of the key ideas leading to SPIPA and both papers were facilitated by the inspiring atmosphere of one of my favourite places on the Cary Institute’s campus, the Fern Glen (Figure 7.1).
FIGURE 7.1: The Fern Glen, an inspiring place on the campus of the Cary Institute of Ecosystem Studies in Millbrook, NY, USA. Many times I enjoyed sitting on one of the chairs (on the right side of the picture), watching, smelling and listening to the Wappinger Creek (on the left side of the picture) and other elements of the fascinating nature around me – and several times I returned to my desk enriched not only by the joy of nature, but also by a thought that turned out to be useful (e.g. Jeschke et al., 2012b). I took this picture in the fall of 2010.
7.1 What the paper is about: zombie ideas and the hierarchy-of-hypotheses (HoH) approach
The idea behind the paper was to synthesize available studies on key hypotheses in the field of invasion biology and see how the hypotheses’ empirical support has changed through time. It goes back to my days as a student that I was often wondering to which degree established hypotheses and concepts that we learn as students from teachers and as researchers from colleagues are actually empirically supported. Identifying what I later learned can be called zombie hypotheses was a key goal of the paper. Jeremy Fox (-Fox (2011), based on ideas in Quiggin (Sun, 05/06/2012 - 12:00)) described zombie ideas as follows:
“Ideas, especially if they are widely believed, are intuitively appealing, and lack equally intuitive replacements, tend to persist. And they persist not just in spite of a single inconvenient fact, but in spite of repeated theoretical refutations and whole piles of contrary facts. They are not truly alive – because they are not true – but neither are they dead. They are undead. They are zombie ideas.” —Fox (2011)
Zombie hypotheses are hampering the progress of science and can lead to ill-informed policy decisions. It is thus important to identify them. At the same time, researchers don’t necessarily want to hear that their favourite idea is a zombie and should be buried. I will come back to this point later in lessons learned, when reflecting on why it was hard to get the paper published.
Another key aspect of Jeschke et al. (2012b) was how to structure the many empirical tests that have been published on our focal hypotheses. A challenge here is that broad hypotheses are typically not tested in their whole breadth. Instead, a given empirical test usually addresses a specific formulation of a broad hypothesis, and this specific formulation can be called a sub-hypothesis. We realized this was the case for the major hypotheses we looked at in our study, and thus needed a tool to structure the empirical studies we found in the literature.
While looking for such a tool and discussing different options, Tina Heger, Sylvia Haider, Anna Pahl (née Liebaug) and I held a workshop in Benediktbeuern near Munich, Germany, titled “Tackling the emerging crisis of invasion biology: How can ecological theory, experiments, and field studies be combined to achieve major progress?” (March 2010). The discussions there and our work on Jeschke et al. (2012b) led to the hierarchy-of-hypotheses (HoH) approach which was presented for the first time in Jeschke et al. (2012b), and then in Heger et al. (2013) as the publication related to the workshop. Since then, we have further improved and applied the HoH approach (Heger & Jeschke, 2014; Braga et al., 2018; Bartram & Jeschke, 2019; Ryo et al., 2020; Heger et al., 2021), which also led to a book (Jeschke & Heger, 2018) and the Hi Knowledge website featuring an interactive conceptual map of invasion biology where the HoH approach is a key underlying tool.
7.2 The hard road to get the paper published
As you’d probably expect in a chapter from a book about the struggles to get papers published, the manuscript was rejected by several journals. After having worked on the manuscript for quite some time and being convinced that we produced a great piece of work, we started our submission process in early 2011 with the most prestigious journals, Science and Nature. We did not really expect the manuscript to fly into one of them, but we thought it was worth trying. Well, that dream was over very soon, as the manuscript was desk-rejected in both journals.
We then decided to submit to the Proceedings of the National Academy of Sciences USA where the manuscript was peer-reviewed, and then rejected. On the positive side, we received concrete criticisms this time that helped us to improve the manuscript, implementing at least those points we agreed with. One criticism was that we should have applied a formal meta-analytic approach. We partly understood and acknowledged this point, but were disappointed that the novelty of our paper was only appreciated by one of the three reviewers. With two reviews being negative, the editor could only reject the manuscript. I would like to make a little side note here, as there has sometimes been confusion about the relationship between the HoH approach and formal meta-analysis: both approaches can easily be combined (see also Heger et al., 2021), and we have done so (Jeschke & Pyšek, 2018). In Jeschke et al. (2012b), however, HoH was combined with a semi-quantitative scoring approach, and this attracted criticism.
After having been rejected three times, we asked ourselves the “What next?” question and decided not to give up, but send a presubmission inquiry to Trends in Ecology & Evolution (TREE), another high-impact journal. We were happy to receive a positive decision and invitation to submit the full manuscript, which we did. Unfortunately, we received a rejection after peer review in September 2011. The reviews were again mixed, and the sentences we received from the editor were frustrating: “While I’m confident you would be able to respond successfully to the concerns raised, it is clear to do so would require making the article much too close to primary research. That would take your article out of the remit of TREE.”
So we revised the manuscript again based on the points made by the reviewers with which we agreed, and set our hopes on success. The next journal we targeted was Frontiers in Ecology and the Environment. Sadly, though, we again received a rejection after peer review. One reviewer was positive overall, but the other one negative, hence the editor declined the manuscript.
7.3 The happy publication end
After having been rejected five times, we were not ready to give up and revised the manuscript again based on the most recent comments we received. We did give up on submitting to a high-impact journal, though, and decided to send the manuscript to a promising new open-access journal, NeoBiota. Still being convinced about the quality of our study, we thought we could perhaps even support this young journal by submitting the manuscript there.
And this is what happened and led to a happy end of this paper’s story: We submitted the revised manuscript to NeoBiota in May 2012. This time, we received more positive news after peer review. The editorial decision was ‘major revisions’, so we again revised the manuscript based on the new comments we had received and wrote a response letter to indicate how we incorporated each comment. The revised manuscript was accepted by NeoBiota in August 2012. About 1.5 years after the first submission of the manuscript to a scientific journal, it was finally “in press” !
Although it had a very bumpy start, this paper turned out to become influential. As outlined above, it was the first paper featuring the hierarchy-of-hypotheses (HoH) approach and has thus laid the foundation for many other studies. The paper turned out to really support the journal NeoBiota, as we had hoped, being among the most widely read and cited papers ever published in this journal (having been viewed >28,000 times according to the journal’s website and cited >275 times according to Google Scholar in November 2021).
7.4 Why was it so hard to get this study published?
When submitting this manuscript to a journal, we received partly positive and partly negative reviews in most cases. Editors usually reject manuscripts with such mixed reviews, and this was also the fate of our paper. In particular, while some reviewers liked the study due to its innovativeness and novelty, others criticized our semi-quantitative scoring approach. Both a narrative (qualitative) review and a fully quantitative analysis probably would have been easier to publish. Reviewers usually have a certain expectation on which approach to follow for a particular research question, and while some are happy to see a novel approach, others are more critical and focus on the drawbacks of a new approach as compared to the approach they expected (and probably would have chosen themselves); and since all approaches have some drawbacks, such reviewers typically come to a negative recommendation.
Another reason why this paper was hard to get published probably was that its results are rather discomforting. The paper showed that the empirical support of several key hypotheses in the field of invasion biology is relatively low, and have even declined through time overall. This is something we were also quite surprised about, but it might be considered an irritating finding that made it harder to get published.
7.5 What are the lessons learned for publishing papers?
Looking back at this paper, and others I have published, I offer three suggestions on how to deal with rejections of submitted manuscripts.
First, take reviews seriously. It is important to repeatedly read and reflect on reviews you have received on a submitted manuscript. When I receive reviews, I usually have a quick first look at them. If they are negative, they are sometimes hard to swallow. After a night or two when having digested the disappointment, though, it is easier to read negative reviews in a way that allows me to take advantage of them and find the parts that really help improve the manuscript. Some reviews are clearly full of such constructive points. Others not so, but they can still be helpful when reflecting on them. Points raised by reviewers that I don’t agree with are often based on misunderstandings caused by unclear wording, thus realising this also helps improve the manuscript. Don’t immediately submit your manuscript to the next journal after having received a rejection. I recommend taking the time to seriously consider the reviews and thoroughly revise the manuscript accordingly.
Second, be grateful to reviewers. Independently of whether you agree or disagree with the comments made by reviewers, you should be grateful that they took the time to read, comment and assess your manuscript. Further, you should not waste the time of editors and reviewers by submitting a manuscript to a journal where it clearly does not fit. I always feel a bit guilty when one of my manuscripts had to go through peer review with different journals, as several colleagues had to invest substantial amounts of time for it. This was clearly the case for the focal paper of this chapter. In addition to being grateful that others have reviewed your manuscript, you should also serve as a reviewer of your colleagues’ manuscripts if you have the opportunity to do so. You don’t have to do more than your share, but the peer review system only works if (a) we do not waste the time of reviewers and editors, and (b) we invest time ourselves to review and edit our colleagues’ manuscripts (cf. McPeek et al., 2009; Lajtha & Baveye, 2010; Kaushal & Jeschke, 2013).
Third, believe in your study. Some reviews are ground-shaking and devastating. Here again, it is important to think about such reviews and reflect on your study: Are these reviewers right? In many cases, you will find that while you can follow some critical points made by reviewers, devastating reviews typically ignore the positive aspects of a paper. Taking our focal paper here, while some reviewers had expected a fully quantitative meta-analysis and criticised our paper for the semi-quantitative approach we took, they largely ignored the positive aspects of the study. We were still convinced of the central ideas in our paper and thus didn’t give up on it. I recommend that you believe in your study, to remember why you did it in that particular way. It can be, of course, that the reviewers found a fatal flaw that you were not aware of before and that this cannot be corrected, as it lies at the basis of the study. In such a case, the paper should indeed be abandoned, and you can be grateful that the reviewers spotted that fundamental flaw. Most studies are worthy of being published, though, and it is important to get them out there, so that others can benefit from them. Not publishing a study means you and your co-authors are the only ones with access to it, which is against the idea of open science and leads to an increase in dark knowledge (Jeschke et al., 2019). Therefore, being persistent and believing in your study helps you as well as helping science and society at large.