More Resources

Author Resources Center

Share this resource


    This Resource

    Version 1
    Published 26 January 2016

    PDF Version

    PDF Checklist

    Assessing Open Access Journals: A Full Guide

    An increasing number of scholars want to publish in open access journals. There are clear benefits to publishing under an open access model, such as wider dissemination and better visibility. However, open access publishing presents some pitfalls. This practical guide helps you to navigate the world of open access publishing and publish your papers in reputable open access journals. Note that some of the Ediqo editing packages include personalized advice to find a good journal for you. For your convenience, we have provided a quick checklist at the end of this article.


    Many scholars have a growing interest in publishing in open access journals. There are clear benefits of publishing in an open access journal: papers available under open access are immediately and widely available, which benefits researchers in economically less developed regions, as well as teachers and students, hospitals and doctors, the media and society in general. Further, the Creative Commons license means that authors retain copyright of their paper, and permits other scholars to easily build upon their published work. This leads to increased dissemination and usage of research papers and better visibility for their authors. Ultimately, open access papers reach a wider audience and have a higher average impact, an effect known as the Open Access Citation Advantage.

    Nevertheless, open access publishing has its dark side with a number of publishers being called into question on their commercial interests versus their true motivation to advance the communication and dissemination of research and science. Jeffrey Beall, a Librarian at Auraria Library, University of Colorado Denver, has been on the lookout for what he terms “predatory publishers”. He has been exposing the shady practices of some publishers in his blog Scholarly Open Access since 2012. It is important to keep in mind, though, that Beall is an open critic of open access publishing. Some other librarians have recently joined forces to establish the Quality Open Access Market (QOAM) initiative – a database scoring journals through academic crowd sourcing. The database currently scores approximately 3000 journals, just under one third the number of journals listed in the Directory of Open Access Journals (DOAJ).

    To help scholars identify reputable, quality open access journals, we have prepared this guide. Note that most checks can also be applied to non-open access journals or hybrid journals (journals available to subscribers only but where authors have the choice to make their article open access by paying an additional fee). The guide helps you to examine the open access status, indexing database coverage, archival policies, publication statistics, and editorial policies of a journal. Unfortunately, there is no single indicator for a journal’s reputation, rather reputation is built upon over time based on a range of positive factors.

    Finding an Open Access Journal

    First and foremost you need to identify a few journals which are relevant to your research. You probably already have a habit of regularly reading and citing papers from some journals as part of your research. Those are usually the journals you would consider as possible publication venues first. They may not necessarily be open access journals, though. To check the open access status of a journal, see if the journal is included in the Directory of Open Access Journals (DOAJ). Note that hybrid journals are not covered in DOAJ, which is limited to pure open access journals that publish under permissive Creative Commons licenses. Furthermore, there is the Open Access Scholarly Publishers Association (OASPA), a membership-based industry organization regrouping publishers of open access journals. OASPA members publish at least one fully open access journal. Alternatively, you may browse DOAJ by subjects. A DOAJ subject search will also reveal a number of scholar-run journals that may not be represented by a publisher-member of OASPA.

    Indexing Database Coverage

    Any reputable journal is indexed by at least one of the major indexing and abstracting services, namely:

    To a lesser degree:

    There are numerous other specialized indexing services (Agris, Geobase, PsycInfo, MathSciNet, to name a few) that cover niche areas. However, any journal important to a niche area is very likely to be indexed by at least one of the above services as well. We recommend that you check the indexing status of a journal against the above databases. Start by checking those databases that yourself use most frequently. To make it easier, you may also check your journal’s performance in Sciforum Statistics, a database crossing records from CrossRef (the DOI registration agency) with lists of covered journals from the above indexing services. If you cannot find your journal in Sciforum Statistics, this will most likely mean that the journal in question is not even using DOI numbers. DOI numbers are the de facto standard to permanently link scholarly papers. We do not recommend checking database coverage on the journal homepage itself, as some journals may publish bogus information.

    Note that it usually takes several years for new journals to be covered by any of these indexing services. If a journal is not indexed, it does not necessarily mean that the journal is of low quality. In cases where you consider submitting to a relatively new journal, it is advised to check if the publisher of that journal has older, more well-known journals in these indexing databases. If you personally know some of the editors of the journal, you should send them a short e-mail to confirm their board membership (by indicating the journal name and the journal homepage that you found) and ask them whether they recommend you to submit your paper to this journal. It has happened in the past that some journals have listed a few editors on their homepage without approval of the editors in question, or even listed fake profiles. Getting a written confirmation by e-mail from the institutional e-mail address of one of the editors will support the validity of a journal and the commitment by its editors: the reputation of the editors is linked to the reputation of the journal.

    Is a journal covered in Thomson Reuters’ products?

    Note that only journals covered by Thomson Reuters’ Science Citation Index (SCI), Science Citation Index Expanded (SCIE) and the Social Sciences Citation Index (SSCI) receive Impact Factors. Thomson Reuters’ flagship product Web of Science may—depending on the subscription of your university—include other journals from the Arts & Humanities Citation Index (AHCI), Conference Proceedings Citation Index (CPCI), Current Contents, INSPEC, BIOSIS Previews, and other databases. Therefore, search the Journal Master List by journal title or ISSN number and specifically check if the journals is covered in either one of SCI, SCIE and SSCI. If you cannot find the journal, it is not currently covered. Thomson Reuters' Journal Citation Reports, in which the Impact Factors are published once per year, include a publicly available notice of journal titles that were suppressed from receiving an Impact Factor, usually for excessive self-citing or citation stacking.

    Is a journal covered in PubMed or MEDLINE?

    Search the NLM Catalog by journal title or ISSN number. If there is more than one search result, find the right journal from the results and select it to check if the journal is listed as „In PubMed“ or „In MEDLINE“. If you cannot find your journal, it is not covered. Note that MEDLINE is much more selective than PubMed, although MEDLINE is typically accessed as part of PubMed. PubMed also includes all journals that were approved for depositing their full-text to the PubMed Central archive.

    Is a journal covered in Scopus?

    Download the current Journal Title List (a Microsoft Excel file) and filter the “Source Title” column for the journal title you are looking for. The “Coverage” column should indicate “ongoing” for your journal. If you cannot find the journal, or the coverage is not “ongoing”, it means the journal is not currently covered.

    Is a journal covered in INSPEC?

    Download the INSPEC Journal List and search for the journal title or ISSN number. Only titles featured on the list are currently indexed.

    Archival Policies

    To keep publishing costs down, open access journals are typically published electronically on the Internet only. Thus, published content is only available for as long as the journal’s website is online. To avoid that your published research paper disappears from the Internet in case of bankruptcy of the publisher or other circumstances, check the archival policies of an open access journal.

    To this end, you may visit the Keepers Registry. The Keepers Registry crosses archival records from several large archives and national archives, including, LOCKSS (“Lots of Copies Keep Stuff Safe”), CLOCKSS Portico, the US Library of Congress, the Dutch e-Depot or the British National Library.

    However, the Keepers Registry is not extensive. For instance, one of the most important and publicly accessible journal full-text archives in biomedical sciences, PubMed Central, is not referenced in this registry. Also, there are numerous national archives and national libraries, such as the Swiss National Library, the French Archives Nationales, or the German Bundesarchiv, that do not make their holdings available to the Keepers Registry. You may thus also search the Internet for the website of the national archives of the country where the journal is being published and check their records. For instance, all electronic journals and eBooks published in Switzerland are legally required to be deposited to the Swiss National Library e-Helvetica archive.

    Publication Frequency and Journal Statistics

    Besides indexing database coverage and archival arrangements, it is always good to get a better picture of a journal by looking at some key statistics. To establish a regular readership, a journal has to publish a certain amount of content and regularly publish new issues. Look at the table of contents of the current and last volume of the journal to get an idea of how many issues the journal publishes within a year, and how many papers the journal publishes in an average issue. Journals with a regular readership will typically publish 10 items or more per issue, and publish at least quarterly issues (4 issues per year). It is a good sign if the journal invites readers to join a mailing list to receive announcements of new issues: newly published issues will receive immediate exposure through the announcement.

    Beware of journals that are already publishing their third volume or more, and publish less than 10 items per year, or publish issues irregularly. It signals the journal is not taking off—most likely the result of poor management, low editorial standards, or lackluster marketing and promotions. To review the number of papers published annually in a particular journal, you may again look at Sciforum Statistics (Note: as of writing of this article, the 2015 numbers seem still incomplete).

    When looking at the issues, make sure the journal is publishing a high proportion of research articles (sometimes, also called original papers). It is usually more difficult for new journals to attract original research. Look at the first few and last few papers of a volume to see if the journal’s chief editor has published an editorial. An editorial may give important information about the strategy and editorial policies of the journal, and shows the editor's commitment to the journal.

    If the journal has already published a few volumes, you may also compare the number of editors on the editorial board of the journal with the number of papers published per year. The fact that the journal has, say, 80 board members, but publishes only 15 papers per year, may hint that the editorial board is bloated and has more of a promotional than an editorial role. To work efficiently, an editorial board for a new journal should comprise between 10 and 30 board members. However, well established, large journals may have larger boards, typically divided into sub-disciplines of the journal’s subject.

    For print journals (usually not applicable to online or open access journals), you may want to check circulation figures. Circulation is the number of printed copies or subscriptions to the journal. The higher the circulation of a journal, the wider the readership. Open access journals are typically not printed. Nevertheless, an increasing number of online and open access journals publish statistics pages to give an impression of their readership. Such statistics typically include the number of page views for a journal, the number of PDF or full-text downloads, and, sometimes, a geographical breakdown of the visitors.

    Editorial Policies

    Good journals have a clearly defined and reasonably narrow scope. Look for the “About” or “Scope” page of the journal: it should give a short overview of the covered subjects, usually complemented by a listing of the main keywords. Compare the scope of the journal with a few of the published papers. If you find papers that are completely out of scope, it may reflect low editorial standards or lack of enforcement of the standards of the journal. If the scope of the journal is excessively broad, or covering several disciplines that usually do not belong together, the journal may simply be trying to attract as many papers as possible, including substandard ones.

    Make sure that journals that claim to have an international scope are truly international: the editorial board as well as the authorship should be geographically diverse.

    Look for the editorial board of the journal. The journal should clearly identify an academic chief editor (an editor in charge of the academic standards of the journal, usually called Editor-in-Chief). This should typically be a full university professor or a senior scholar from a research institute. The journal may list in-house editors as well, but make sure that they are in line with academic editors. Good journals also clearly list the publisher’s name and full address in the imprint (the publisher may be an organization).

    Any reputable journal clearly outlines their peer-review and editorial process. Look for the journal’s guide for authors (or instructions for authors): it should describe the peer-review and editorial process, and give detailed instructions on how authors are expected to prepare and submit their paper.

    Look for a publication ethics statement, which will detail ethically unacceptable behavior, such as plagiarism, image and data manipulation, etc. and describe the criteria for authorship, rules relating to ethical approval of use of animals or humans in research, and instructions to register clinical trials. Some journals also include links to the guidelines of the Committee on Publication Ethics (COPE), or state their COPE membership. You can search COPE for journal and publisher-members.

    You may also want to check previously retracted papers from a journal by browsing Retraction Watch. Retraction Watch is a blog setup by Adam Marcus and Ivan Oransky in 2010, which reports and investigates retractions from scholarly journals and cases of scientific misconduct. Retracting a paper is not necessarily bad for a journal: if it made the mistake of publishing an erroneous paper, or was duped by a paper’s authors, it should take on the responsibility to investigate and correct the scientific record when appropriate. It seems, however, that many journals’ editors and publishers are reluctant to retract papers, do it very slowly or, sometimes, do so without providing a rationale for the retraction.

    If you are considering submitting your paper to an open access journal, remember that journals often rely on publication fees or “article processing charges” (APCs) to finance their operations. Although several studies found that more than half of all open access journals listed in DOAJ do not impose author charges, the reality is that many of the most reputable open access journals and almost all open access journals by commercial publishers do have such charges. Those fees or charges should be clearly stated on the journal homepage or in the guidelines for authors. Journals that try to hide their fees may not be a good choice for your paper: they may hide other facts as well, such as a lack of proper peer-review.


    Decent journals invest in their technology inventory to satisfy the increasingly sophisticated demands of their author- and readership. Make sure that the journal of your choice offers most of the following features:

    A further check you can conduct is to type the journal name into Google: a reputable journal will rank high in search engines. A high Google ranking indicates an abundant number of links pointing to the journal and its content, and that the journal has mastered Search Engine Optimization (SEO) techniques. If the journal ranks high, you have a good chance that your own paper will rank high in search engines once it is published.

    Checklist: Positive Signs

    Quality Indicators Specific to Open Access Journals

    Quality Indicators Applicable to All Journals

    Version 1 (26 January 2016). This guide was written by Dietrich Rordorf. Alistair Freeland commented on a draft version. Joanna Schuurman did thorough proofreading. For questions, please send a message to Dietrich Rordorf, E-mail: Refer to the Resources section on the Ediqo website for more guides and checklists.

    © Ediqo LLC (, 2016. This article is published under a CC-BY-NC-ND license, which permits reproduction of the entire article, without alteration, for non-commercial purposes.

    Share this resource