Changing trends in otorhinolaryngology publishing| ACTA Otorhinolaryngologica Italica

Abstract

The aim of this study is to compare the changes in impact factors and citation numbers of Open Access (OA) vs subscription-based (SB) journals between 1999 and 2016 and to explore the changing trends in ORL publishing. All data extracted from SCImago Journal and Country ranking (SJR) website have been used as input for statistical analysis. The chi-square test of independency was applied in order to understand whether the ratio of number of OA journals of ORL category have dramatically changed between years 1999 and 2016. Also, the years and impact factors of journals belonging to the OA and SB journals have been graphed separately and the changes of annual SJR ranks of both journal types have been compared using one-way Z-test. There was a statistical difference as the proportion of OA Journals were not equal to the proportion of SB Journals throughout the years 1999 and 2016, and it showed the tendency to increase greater compared to SB Journals (p < 0.01). Although the overall level of impact factors of SB journals was generally high, by comparing two regression models, it was obvious that the level of increase of the impact factors of OA journals were significantly higher (p < 0.01). When choosing where to publish, it is important to consider the journal’s visibility, cost of publication, IF or SJR of the journal and speed of publication as well as changing trends in medical publishing nourished by the Web of Science.

Introduction

Open access (OA) means unrestricted, free access to all scientific information, although its significance and impact are yet not fully understood. With the transformation from analogue, printed to digital electronic media for dissemination of scientific information in the last decade, the concept of OA publishing has become even more prominent and the market for scientific publications is currently more global and heterogeneous. The potential of technical reproduction and dissemination worldwide provided by the Web of Science have enabled new business models for scientific publishers, where anyone with internet access can read OA articles, and the required resources to operate such medical journals are supplied by means other than charging readers ¹. Thus, OA is not only a single unambiguous term, but rather a set of possible strategies for distributing unrestricted scientific information accessible to all with various publishing modalities such as green, gold, platinum etc.

The idea of a journal that is fully accessible to the public, with no financial barriers, seems theoretically great, but when it is time for publication and journal selection, many researchers may find difficulty in making a decision between an OA or a traditional subscription-based (SB) journal. In the early days, OA journals were fraught with doubts and scepticism about the reliability and peer-reviewing quality, thus publishing in such journals was evaluated as a “quick fix” for academic promotions. However, over the years, major reliable publishers such as PLoS and BioMed Central have taken up this type of publishing modality and OA publishing started to gain repute and began to be indexed in quality databases ². A study in 2012 scientifically showed that OA journals began to reach same scientific quality and impact just like their SB equivalents, and the average citation rates were only 30% higher for subscription journals at the time of publication ¹. A recent study has shown that ORL journals with active social media accounts such as Twitter had much better visibility and significantly higher academic influence and H-index ³. In a similar manner, since 2012, it is our personal observation that there is even a greater increase of citation numbers for the ORL articles published in OA journals, which can be attributed to the widespread usage of the internet and smartphones globally after this date. The aim of this current study is to test this hypothesis and compare the changes in impact factors and citation numbers of OA vs SB journals over years, thus exploring the changing trends in ORL publishing.

Materials and methods

In this study, two statistical methods are used to evaluate the changing trends in publishing in ORL journals. All data extracted from SCImago journal and country ranking (SJR) website have been used as input for statistical analysis. Initially, we applied the chi-square test of independency to understand whether the ratio of number of OA journals of ORL category listed inside SJR website have dramatically changed between years 1999 and 2016. As a second analysis, data of yearly impact factors of each ORL journal have been taken from the same website as presented on 20 December 2017 and listed separately for OA and SB journals using MINITAB Statistical Software (Release 17, Minitab Inc. State College, PA, USA). The years have been taken as a discrete factor variable for each journal and the annual SJR have been taken as response variables. By using a statistical package, the years and SJR of journals belonging to the OA and SB journals have been graphed separately and the ranking trends of both journal types have been compared. This comparison has been done by comparing the slopes of the two lines that are fitted to the two different data sets by least square method. The slopes of the two regression lines have been compared by a one-way Z-test and statistical significance of the difference of the trends was analysed.

Results

As of 1999 among 82 ORL journals, only 3 were OA journals. However, in 2016, 105 journals were published on the SJR website and 18 were OA. The increase of proportions may be shown by a chi-square of independency. When we arrange Table I regarding the expected values of for the procedure, Table I becomes:

The hypothesis should be:

1. H₀ = p₁ = p₂ (the proportion of the number of OA and SB journals are the same through years 1999-2016);
2. H₁ = p₁ ≠ p₂ (the proportion of the number of OA and SB journals are not the same through years 1999-2016). When we calculate the chi square test statistic

The critical χ² table value for αÊ= 0,05 for (2-1) x (2-1) = 1 degree of freedom is equal to 3.841. Thus, we can conclude that: the hypothesis that says the proportions of number of OA journals through years are equal to the proportion of the number of SB journals can be rejected. Thus, there is a statistical significance that the proportion of the number of OA Journals are not equal to the proportion of SB Journals through 1999 and 2016. The proportion of OA Journals show a tendency to increase more compared to SB Journals (p < 0.01).

In addition, the changes of annual SJR of two types of journals shows a different trend through years between 1999 and 2016. For each year, the data have been split into two categories as SJR of OA journals and non-OA journals. Starting from 1999 to 2016, the impact factor of all journals was entered into MINITAB and two separate graphs were drawn (Fig. 1A, B).

Although the general level of impact factors of SB journals is generally high, by comparing the two graphs it can be seen that the level of increase of the impact factors of OA journals are significantly higher. Although the increase of the impact factors of OA journals seems better, a one-way Z test was performed to compare the trends. The blue lines that passes through the data are the regression models that are fit to the data by the least square method. The slope coefficient of the first model that represent the subscription-based journals is 0.01126 and the second representing the OA journals is 0.02104. The standard error of the slope coefficient of the first model is s_b1 = 0.00202 and the standard error of the slope coefficient of the second model is s_b2 = 0.00311. The equality of the two slopes may be tested by the following one-way Z test.

H₀ : β₁ = β₂ The slopes of the models are equal

H₁ : β₁ ≠ β₂ The slopes of the models are not equal

Since z value is higher than the critical value for which is equal to 2.33 for a one way z-test, which means the prob value is lower than 0.01 (p < 0.01) the null hypothesis may be rejected with a confidence level of 99%. The slopes of the two lines formed by the least square regression model are significantly different. The slope of the model that shows the increasing trend of OA journals is greater than the slope of the line that is fitted by the subscription based journal data. The slopes of the two lines are significantly different. It can be seen that the SJR of OA journals are tending to increase more rapidly, especially in the recent years.

Discussion

Since the publication of the first scientific journal, the Philosophical Transactions of the Royal Society, in 1665, today’s journal publication landscape has exploded to more than 28,000 active scholarly peer-reviewed English-language journals, sharing the scientific discoveries of seven to nine million scientists in academia and industry. As of 2013, the scientific publication niche has been defined as a “bottomless pot of gold”, with an estimated 110,000 employees globally and $ 25 billion annual revenue from public sources (via SB journals) and from the authors themselves (via OA journals). The market was predicted to grow at about 4% annually through 2017 ⁴. This is not an astonishing fact, given that scientific effort has never diminished, and with the fierce competition for academic positions, scientific publication rates are likely to increase further. However, this last decade has seen the growth of OA journals that offer a slightly different concept of peer-reviewed scientific publication to more traditional SB journals ⁵.

The “h-index”, a new indicator proposed by Jorge E. Hirsch in 2005, is being widely used today to evaluate scientists’ research performance, rather than just the number of articles they produce, and measures productivity of the researcher and the quality of that productivity taken together. In order to have a high h-index, it is necessary to publish a significant number of articles and each should have high citation numbers. At this stage, easy accessibility and high visibility of a printed article plays a major role in achieving the desired high number of citations. Although OA journals are fulfilling these criteria in an appropriate manner, they have been criticised for having a lower impact factor and poor peer review quality for many years. However, the results of this study show that these determinations have begun to change with an increasing ratio of OA journals in scientific indexes.

The “Impact Factor” (IF) is the major indicator of scientific importance of journals, calculated annually by Institute for Scientific Information (ISI) and by definition in any given year is the ratio of the number of articles cited all citable documents published in the two previous years to all citable documents in the same period of time ⁶. The SCImago journal rank (SJR) indicator is a novel alternative index of scientific influence of the academical journals using the Scopus database. The SJR is defined as a size-independent prestige indicator that ranks journals by their “average prestige per article”. It is an assessment of scientific influence of journals that accounts for both the number of citations received by a journal and the importance or “prestige” of the journals where such citations come from SCImago journal and country ranking website ⁷. Based on the comparisons made by Falagas et al. ⁸, the SJR index might be a serious alternative to the well-established journal IF, based upon its OA nature, larger source database and assessment of the quality of citations, and it is recommended that the authors should consider all of these indices rather than just IF alone in assessing the influence and importance of medical journals in their respective disciplines ⁹. In the present study, otorhinolaryngology journals were selected from the journal ranking section of SCImago journal and country ranking website ⁷, where they are grouped as quartiles (Q1-4) and as a result of the regression model applied; it was obvious that the level of increase of the impact factors of OA journals were significantly higher than SB ones. As of 2016 SJR listings, we have started to encounter more OA journals in the indexes and this observation was also statistically significant.

Conclusions

When choosing between OA and traditional journals, it is important to consider the journal’s visibility, cost of publication, IF or SJR of the journal and speed of publication. To the best of our knowledge, this is the first study in the English literature to compare the OA and SB journals in the ORL field, demonstrating novel trends in medical publishing between 1999-2016. Nevertheless, it is still recommended to consider all the pros and cons of both journal types to achieve the desired high-citation numbers.

Figures and tables

Figure 1.(A) The SJR of SB Journals of ORL during 1999-2016; (B) The SJR of OA Journals of ORL during 1999-2016.

References

1. Björk BC, Solomon D. Open access versus subscription journals: a comparison of scientific impact. BMC Med. 2012; 10:73. DOI
2. Cuschieri S. WASP: is open access publishing the way forward? A review of the different ways in which research papers can be published. Early Hum Dev. 2018; 121:54-7. DOI
3. Wong K, Piraquive J, Levi JR. Social media presence of otolaryngology journals: the past, present, and future. Laryngoscope. 2018; 128:363-8. DOI
4. Ware M, Mabe M. The Hague: International Association of Scientific, Technical and Medical Publishers; 2015.
5. Guttman C. Open access vs subscription-based scientific publications. 2013. Publisher Full Text
6. Ramin S, Sarraf Shirazi A. Comparison between impact factor, SCImago journal rank indicator and Eigenfactor score of nuclear medicine journals. Nucl Med Rev Cent East Eur. 2012; 15:132-6.
7. SCImago journal and country ranking website.Publisher Full Text
8. Falagas ME, Kouranos VD, Arencibia-Jorge R. Comparison of SCImago journal rank indicator with journal impact factor. FASEB J. 2008; 22:2623-8. DOI
9. Rizkallah J, Sin DD. Integrative approach to quality assessment of medical journals using impact factor, eigenfactor, and article influence scores. PLoS One. 2010; 5:e10204. DOI