POSTER: Development of a New Algorithm and Impact Metric for Scientific Journal Impact

POSTED BY

JOSEPH B. LAUDANO, BS PHARM, PharmD

SHARE

The Problem With JIFs…And Our New Solution

Click here for full poster

Traditional journal impact factors (JIFs) may not be the best way to gauge the impact of specific journals but, until recently, they provided the best metrics available. The problem – and the cause of years of discussion within the scientific community – is that traditional JIFs are used as surrogate measurements to determine journal impact in lieu of more specific metrics.

Traditional, citation-based JIFs are affected by multiple factors. A journal’s JIF may be enhanced by a few high-profile articles that result in a flurry of citations, although lower ranked,more targeted, journals may publish more primary research. According to one study, between 15 and 25 percent of the articles in the journals it studied accounted for 50 percent of the citations from those journals[i]. Another study found that basic research is cited three to five times more often than clinical research[ii].

To further skew the data, researchers’ choice of citations often is based upon their available resources. There often is a two-year lag time between article submission and publication in major journals, and their acceptance rates may be quite low. In addition, important journals aren’t always integrated into leading lists, so those journals – especially when they are from developing regions –don’t necessarily appear in the rankings.[iii] Medmeme’s new journal dynamic impact factor algorithm, for example, identified highly ranked journals that appeared in networks unfiltered lists, but which did not appear in their filtered lists[iv]. Consequently, MA professionals are missing opportunities to deliver information directly to the most relevant audiences. For example, although having one’s article accepted for publication in the Lancet or the New England Journal of Medicine is prestigious, only a fraction of its readers have occasion to put any specific article to use. That article may have more impact in a tightly targeted, but perhaps lesser known scientific publication, or may benefit from immediate dissemination in an open access journal.

In that environment, determining exactly which publication will have the most impact is challenging. The scientific community, well aware of the challenge, is acting. Some of the premier scientific organizations throughout the world have made public statements to guide scientific bodies in their use of JIFs. In 2012, the editors and publishers of well-respected scientific journals joined together to create the Declaration on Research Assessment (DORA)[v]. Their key message was that multiple article-level metrics should be used to assess scientific contributions.

A New Algorithm for an Unmet Need

To address the need for better metrics, Medmeme developed a new algorithm that assesses the impact of four key areas that are important to Medical Affairs professionals:

  1. Journals
  2. Conferences
  3. Authors
  4. Presenters

Basically, the new algorithm determines the dynamic impact factor for specific journals, conferences, authors and presenters, thereby allowing scientific communications to be targeted more accurately than ever before.

The Journal Dynamic Impact Factor (JDIF) uses Eigenfactor as a baseline, and integrates the qualitative metrics of journal type, journal quality score, article type, therapeutic area, and the age and therapeutic areas of the articles published in scored journals.

More specifically, the Medmeme algorithm collects and totals the past three years of Eigenfactor scores for each journal and normalizes those scores to calculate the journal weight. Then it searches the entire Medmeme database of abstracts, retrieves those relevant for the therapeutic area or specific disease in question, and tags and scores each abstract as coming from either a journal or a conference. Next, it calculates the journal article score for each article in that journal, and totals the score to determine the journal impact.

By going further than Eigenfactor, this algorithm identifies important, relevant journals that may not be discovered when using only conventional methods (like Eigenfactor, SCImago Journal Rank (SJR), and h-index).

The Test

 

Using Medmeme’s JDIF algorithm,12 journals with content in MS not previously included in the top 20 EF, SJR, or h-index lists moved up in position, and 13 journals within the top 20 JDIF list were identified that were not included in either of the three traditional lists (complete listing) when filtered under Clinical Neurology. This suggests that Medmeme’s Journal Dynamic Impact Factor scoring is significantly different than traditional, citation-based journal impact scoring, thus expanding audiences and opportunities by identifying journals with the most impact and relevance for specific types of articles.

The difference was because, as DORA advocates, multiple article-level metrics were incorporated into the algorithm. As a result, scientific communications can be targeted to the journals that have the most impact for that specific subject, thus enabling them to reach the most appropriate audience. For MA professionals, accurately selecting the scientific journals with the most impact for specific scientific submissions has just become easier.

 

[i]Larivière et al.(2016) A simple proposal for the publication of journal citation distributions. doi: http://dx.doi.org/10.1101/062109.– Publication of Journal Citation Distributions.

[ii]Bornmann, Lutz, Pudovkin A. (Dec. 19, 2016) The Journal Impact Factor Should Not Be Discarded. J Korean Med Sci. published online doi:10.3346/jkms. Print 2017.32.2.180.

[iii]Cantín, M, Muñoz M., Roa, I. Comparison between Impact Factor, Eigenfactor Score, and SCImago Journal Rank Indicator in Anatomy and Morphology Journals.Int. J. Morphol.33(3):1183-1188, 2015

[iv]Laudano, JB, Skirbe, PA, , Hong, SA, Wei, GA, Matheis, RB. Development of a New Algorithm and Impact Metric for Scientific Journal Impact  Poster Presented at the 14th Annual Meeting of the International Society for Medical Publication Professionals; National Harbor, MD; April 30 – May 2, 2018

 

[v]San Francisco Declaration on Research Assessment https://sfdora.org/read/Page accessed May 8, 2018.

[vi]

Int. J. Morphol.,

33(3):1183-1188, 2015.

Comparison between Impact Factor, Eigenfactor Score,

and SCImago Journal Rank Indicator in Anatomy and

Morphology Journals

The availability of research material to scientists and researchers determine their pattern of citation; some references are not available for many scientists due to limitation access and the  indiscriminate use of PubMed

(Manterola et al., 2014), allow lacks integration with all the journals indexed in WoS or Scopus; or the SciELO Network, of particular interest to many developed and developing countries, actually monitored by Thomson Reuters (Cantín, 2014). Journals indexed in PubMed usually have high visibility (Ramin & Sarraf Shirazi), and among Anatomy and Morphology journals indexed in WoS, Advances in

Anatomy Embryology and Cell Biology, Acta Zoologica and International Journal of Morphology are not being indexed…

 

ISMPP POSTER #18:

TAGS

SHARE THIS POST

Medmeme Blog Footer banner
Medmeme Logo
1359 Broadway, Suite 600
New York, NY 10018
Ⓒ Medmeme, LLC 2017. ALL RIGHTS RESERVED