The case for a regional approach to publication impact
Stevan Bruijns and Camillo Lamanna
Faculty of Health Sciences University of Cape Town, Anzio Road, Cape Town, 7935, South Africa
Corresponding author: Stevan Bruijns. E-mail: stevan.bruijns@uct.ac.za
Abstract
Healthcare-related research is largely regional. Put simply, this is because disease burdens differ between world regions. Even global burdens, such as ischaemic heart disease and cancer, display distinctive characteristics in certain regions that are not seen in others. Regional differences in infrastructure, resources and human capital further compound the differences seen, as they affect the way in which the local scientific community can interact with the local disease burden. As such, it seems fair to assume that healthcare-related research ought to be regionally distributed. Although translation of research between regions can sometimes be done, the larger the gap in infrastructure, resources or human capital between regions, the less likely it is that it can be adequately bridged. A recent example of this pertains to accepted life-saving treatment for sepsis in high-income settings, which had the opposite effect when implemented and evaluated in low-income Zambia. This regionality of clinical medicine is, however, not reflected in academic publishing; the impact of a journal is measured and understood by metrics that use the world as their denominator. Therefore, top medical journals are perceived to be relevant equally to all contexts and regions. However, there is a strong case to be made that this lack of granularity is deleterious, and that the creation of a regional impact metric would place clinicians, researchers, and libraries in a better position to understand which journals are relevant to their context and practice.
Keywords: scholarly publishing, low-income population, impact metric
Copyright: © the authors; licensee ecancermedicalscience. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Published: 29/01/2018; Received: 16/10/2017
Healthcare-related research as we know it today started out during the mid-seventeenth century with a distinct focus on the individual. [1, 2] The preferred publication format at the time was the case report (ironically the least preferred today). [3] However, an increased understanding of pathophysiology followed during the mid-nineteenth to early twentieth centuries. This led to the growing recognition that differing underlying disease processes were responsible for the various pathologies identified in case reports. The result was the establishment of various specialist journals, with a focus shift from the individual towards pathology. [3] Then, during the twentieth century, development of newer medicines to treat pathologies—combined with novel evaluation techniques (such as the randomised controlled trial)—shifted healthcare-related research from pathology towards therapy. In those early days, the truths of medicine were largely considered universal, a belief that likely originated during the Enlightenment (eighteenth century) which proclaimed that all men were equal. [4] In other words, if given the same therapy, for the same disease, the response surely should be equal too. Healthcare journals were considered on a par with the natural science journals alongside which they were born. [2] Although the latter did convey universal, region-independent facts about the natural world, the former were unable to do so. The philosophical belief in universal equality simply did not extend to equality of response to treatment. During the second half of the twentieth century, evidence suggested time and again that what worked in one context did not always work so well in another (the Zambia sepsis study is a more recent example). [5] Given growing disparities in health resources, what works in a high-income context is not necessarily feasible in lower income contexts—or the majority of the global population.
While there are good reasons to consider healthcare-related research in context, the current architecture of healthcare publishing is such that findings are presented as global. Arguably, this would be less of an issue if the research outputs of low- and middle-income countries were on a par with those of high-income countries. If Tanzanian-based physicians, say, could practice medicine using an evidence base of similar size and quality to that of, say, an England-based physician, it is unclear what the extra benefit of describing regional research impact would be. However, this is not the situation. There are various reasons why research from low- and middle-income countries fails to assert itself on a global scale. Poor English language skills, lack of experience in applying the research method, lack of funding, are all surely part of the problem. [6] Each of these is a challenge in its own right, none of which is likely to be resolved overnight. However, there are also challenges that can be readily addressed: currently, researchers, academic libraries, and authors from low- and middle-income countries struggle to discern which journals have impact in their respective audiences—a regional impact metric would go some way to rectifying this.
Impact is a challenging theme. Increasingly, researchers have begun to question the hegemony of the Impact Factor, with a multitude of rival metrics suggested in recent years. [7, 8, 9] Moreover, it seems that its basic measure of impact—the citation—does not align well with what researchers in clinical medicine or practising clinicians would consider impact; that is, changing real-world clinical practice. For example, of the ten top-cited articles ever published by The New England Journal of Medicine, three are trials of monoclonal antibody anti-cancer drugs that are largely unaffordable in low- or middle-income countries. [10] Seeing that low- and middle-income countries make up around 82% of the global population, these articles are unlikely to have the global impact suggested by their citations. Therefore, although citations would no doubt contribute to a regional impact metric, it seems unlikely that they would end up as its foundation in the same way as they are the foundation of the Impact Factor.
Another measure, which intuitively appears more reflective of research dissemination, is views. It is a little-known fact that at the individual article level, there is almost no correlation between publication citations and views. [11] Odd as this may seem at first, it does make sense. Scholarly impact (citations) refers to the use of research to generate new research, whereas disseminative impact (views) refers to the use of research to improve knowledge. Some research leans towards a scholarly impact rather than a disseminative impact (and vice versa), but seldom both. As scholarly impact metrics like Impact Factor (citations-per-publication) are far better-known than disseminative impact metrics like views-per-publication, journals with high scholarly impact tend to dominate global rankings. Furthermore, there are tangible barriers to the dissemination of research that are not captured directly by any existing metric. Cost and access are a few notable examples. Cost is an important variable as many low- and middle-income researchers are unable to afford the access charges to subscription articles. Compared to an American researcher, the out-of-pocket expense to access a subscription article is 1.6 times higher for a Brazilian researcher, 2.3 times higher for a Turkish researcher and 3.5 times higher for a Ghanaian researcher. [12, 13] The same ratios apply to the processing charges that allow paid open access publication of regionally important research (also referred to as gold- or hybrid- open access). Although access through self-archiving of articles (also referred to as green open access) provides a low cost access alternative, journals vary widely in their policies on how this can be applied. It seems reasonable to posit that the absence of a disseminative journal ranking has given journals no incentive to increase global access; therefore, one step to addressing access may be to include such a measure within a regional impact metric.
At this point, a journal editor or publisher could say this is all too quick: the function of a journal is to employ a robust mechanism of validation via editorial and peer review to ensure a certain standard of scholarship in what it publishes. This argument is valid but also contains within it inherent hidden biases. We may liken the publication process to a job interview. Prima facie, if an interviewer asks all applicants the same questions and grades everyone on the same scoresheet, then they cannot be accused of bias. But what if some applicants don’t have the training to sell themselves on their CV so never get to interview? What if some applicants cannot afford the fare to get to the hiring office? What if someone from a different background does not use the vernacular expected in interview situations and scores more poorly as a result? These are all hidden biases that are also present in journal publishing which could be mitigated by: offering additional research method and statistical support to researchers from low- and middle-income countries, offering reduced open-access fees to authors from these settings, and offering language assistance to those who do not have English as their first language. Some journals already offer some of these, however it is by no means the standard.
Devising new metrics that respect the regionality of healthcare, and acknowledge the difference between scholarly and disseminative impact, is an important step towards equity in healthcare-related research. Using existing available data to produce regional impact rankings should technically be easy. This may in turn incentivise top (largely Western) journals to adopt some of the aforementioned measures to increase their regional impact in low- and middle-income countries. Differentiating between disseminative and scholarly impact may incentivise journals further to widen access to these countries. Moreover, such metrics can help researchers identify the journals that will provide the highest uptake of their (regional) work, can provide clinicians with the most relevant research in their regions, and enable librarians to save costs by subscribing to a core list of journals that have proven impact within their region. Only a regional view of impact can reveal these nuances and unlock the regional benefits they may present. Newly-launched initiatives such as the Journal Publishing Practices and Standards for journals from the global South has already taken a stab at defining quality on a regional scale. [14] More work is needed to derive, refine and promote reliable regional impact metrics that can help drive quality within non-Western regions.
References
1. Booth CC (1982) Medical communication: the old and the new. the development of medical journals in Britain BMJ 285(6335) 105–108
2. Marta MM (2015) A brief history of the evolution of the medical research article Clujul Med 88(4) 567–570
3. Atkinson D (1999) Scientific Discourse in Sociohistorical Context: The Philosophical Transactions of the Royal Society of London, 1675–1975 (Mahwah, New Jersey: Lawrence Erlbaum Associates)
4. Haakonssen L (1997) Medicine and Morals in the Enlightenment: John Gregory, Thomas Percival and Benjamin Rush (Amsterdam and Atlanta, GA: Rodopi)
5. Andrews B, Semler MW, and Muchemwa L (2017) Effect of an early resuscitation protocol on in-hospital mortality among adults with sepsis and hypotension: a randomized clinical trial JAMA 318(13) 1233–1240
6. Van Hoving DJ and Brysiewicz P (2017) African emergency care providers’ attitudes and practices towards research Afr J Emerg Med 7(1) 9–14
7. Diamandis EP (2017) The journal impact factor is under attack – use the CAPCI factor instead BMC Med 15(9)
8. Butler JS (2017) The evolution of current research impact metrics: from bibliometrics to altmetrics? Clin Spine Surg 30(5) 226–228
9. Rizkallah J and Sin DD (2010) Integrative approach to quality assessment of medical journals using impact factor, eigen factor, and article influence scores PLoS One 5(4) e10204
10. Trends - most emailed, cited, and viewed journal articles at the New England Journal of Medicine [http://www.nejm.org/journal-articles#t=cited – Classic View] Date accessed: 20/11/17
11. Nieder C, Dalhaug A, and Aandahl, G (2013) Correlation between article download and citation figures for highly accessed articles from five open access oncology journals SpringerPlus 2 261
12. Bruijns SR, Maesela M, and Sinha S, et al (2017) Poor local access to African emergency care publications without publisher subscription, a cross-sectional study West J Emerg Med 18(6) 1018–1024
13. De Villiers D, Al Hamzy M, and Megan B, et al (2017) Access to, and cost to access the top 500 cited emergency care publications for researchers without a publication subscription F1000Research 6 1757 [doi:10.7490/f1000research.1114926.1]
14. Journal publishing practices and standards [https://www.journalquality.info/en/] Date accessed: 09/10/17
