Journal Citation Reports have indexed impact factors (NIF, the number of citations registered in year x to articles and reviews published in years x-1 and x-2 divided by the total number of articles in x-1 and x-2) for 21400 journals in 2022, which is almost 10 000 more than in 2021. However, the number of articles with an impact factor greater than 0.2 was invariant at >12 300. The median impact factor, excluding articles with an NIF<0.2, was 2.65 for 2022, which was a huge increase over 2021 at 1.86 and has increased steadily from 1.2 over the last 10 years. Seven journals have an impact factor greater than 100 and they were all in health sciences or life sciences.
DORA (The Declaration of Research Assessment) rejects relying on simple measures like the journal impact factor to evaluate individual and institutional performance and insists that we improve our policies for funding, promotions, and hiring. The presumption has been that evaluators rely too heavily on proxy measures of quality – NIF, h-index (the number of articles, Nart, a researcher has published with at least that number of citations), publisher prestige, and institutional reputation. Certainly, these metrics must carry some weight in an evaluation, but research impact should be underlying measure of quality. However, how does anyone measure research impact: total value of grants, number of students graduated, which then becomes an exercise in counting. In a recent article on the DORA website, Hatch and Curry state that “Most academic reward systems rely on proxy measures of quality to assess researchers” (https://elifesciences.org/articles/58654). In this forum on communication, we must take exception to statements like these as it presumes that DORA has had no impact since 2013 and that proxy measures poorly correlate with contribution. Furthermore, we should suspect most statements that start with the word most unless it is accompanied by experimental data or a reference.
Evaluations are flawed as they rely on individuals who have diverse experience and values. For example, in our chemical engineering design Capstone project, we have as many as 6 people grading oral presentations. Often (a word like most), a project that is assigned the highest mark by one person is give one of the lowest marks by another. This incoherency is discouraging and unexpected as the people marking have the same background and the same grading scheme. The variance in opinions of people assessing grant proposals or scientific awards even be larger.
Impact factors have been increasing and students (and professors and researchers) target journals with the highest impact factor for their work as they perceive that this will have a positive impact on their career. The fact that we target high impact factor journals reinforces the Matthew effect: the rich get richer, or in the case of journals, the good journals get better. So, impact factor must correlate with quality in the long term. In the short term, journals can increase their impact factor by reducing the time from submission to acceptance to publication by accepting poor reviews, for example. Journals that insist on 5 reviewers will certainly suffer in the long term as researchers expect rapid turnaround. Consequently, the higher quality journals that take years to publish articles will eventually be overlooked.