How Do I Know if Grey Literature is Worth Reading?

September 16, 2019/Cora Cole

We get this question all the time. Especially if you’ve never worked in a front-line organization or if you believe that peer review only happens in academia, it can be tough to decide whether it’s worth your while to read grey literature. How do you know that you can trust the processes and conclusions? What are the author’s credentials? What if there are unaccounted biases?

This question—how do I know if grey literature is worth reading—all boils down to grey literature’s accountability. We can further break this question down into an assurance of both content quality and content accountability. Since it doesn’t go through academic peer review, we need to look at alternatives for monitoring grey literature and deeming it worthy. After all, if there’s no academic peer review process to vet out the wheat from the chaff, who holds the authors or organizations accountable?

Before answering these questions, let’s take a moment to look at the peer-review publishing process.

peer review process

Source: Wake Forest School of Medicine

As you can see, review happens after the authors write an article and submit it for publication. Though the authors may be confident the article is complete, this process often opens them up for potentially limitless editing, criticism, and recommended changes. This is time-consuming (24-48 months is the average), frustrating, and distracts researchers from the other work on their to-do lists. It turns out that their work was far from done, even though they thought it was. It can also delay the dissemination of crucial, potentially life-saving information. However, the pressure to publish and their passion for sharing their information forces them to go through this intensive process time and time again. The idea behind all of this additional feedback, of course, is that it creates assurances and accountability for the quality of published content.

So, if grey literature doesn’t go through this rigorous process of anonymous peer reviews, how can we trust it? Is grey literature verified at all?

Well, grey literature’s peer review actually takes place prior to submission for publication. Generally, many departments within the organization look over research and documents before releasing them to the public. Since each department has its own specialized interest, this process protects against undue biases and other scope concerns. Peer review still happens; the only difference is when the review takes place and who those peers are.

Comprehensive Peer Review Process Infograph

In the front-lines, research and information are often the accumulation of the work of several people. It is rare for one person to publish something on behalf of an entire organization or agency, such as Health Canada or the Mental Health Commission. This means that many sets of eyes, each with their own specializations and knowledge, review all publications.

For instance, the committee members who commissioned the work are often the first to review the document. It then may be sent to the organization’s communication department, and from there it could go to the legal department. At the end of the day, this amounts to dozens of reviews for a white paper, policy, or briefing, all conducted by people who are experts at judging content quality and who have a vested interest in the work’s accountability.

Nonetheless, this method of peer review is not without its concerns. Since the onus falls on the author’s organization, GreyLit only publishes content from accredited organizations. Rather than holding a document itself accountable for its content, we hold organizations accountable for their documents. Reputable organizations do not publish misinformation because it would destroy their reputation.

For instance, in order to become a member of the interactive and multi-directional community at GreyLit, one must be a member of an accredited organization or agency in one’s country. Accreditation Canada, for example, has a vetted list of healthcare organizations that we accept as members. Moreover, we target trusted “parent organizations” like hospital networks and organizations like Children’s Healthcare Canada since they can vouch for members within their network. This ensures quality content and a reasonable degree of accountability. Individuals who work on a solo basis are not permitted membership.

In conclusion, grey literature is peer-reviewed. The only difference is that, instead of review happening after the fact, grey literature’s peer review happens before submission for publication. This means that the work is done for the authors once they send it out the door, thus freeing up their time and energy to pursue the next project or publication.

Posted in , ,

Tweet
Share
Share

Topics