Information you can trust

Instead of trust, but verify, you need to question and check the sources before you trust what you hear or read. Too often when discussing a topic people state something as fact by saying “everyone knows” or “I read about it”. These are red flags for anyone listening to or reading something from a source that does not provide the data to back up what they are saying. This is why it is important to check the sources they use. In many cases it’s good to take it a step further and check the articles cited within those sources. In the world of science there is a process that is used to ensure the quality of the information in an article is correct. This is the peer review process.

What’s a journal article?

One of the jobs of scientists and other scholars like historians is to learn and share new knowledge. They run experiments, dig up and read old papers, study people or other animals and more. But for the information that they learn to become “new knowledge,” it needs to be shared. Researchers share this information by giving presentations, making reports, and publishing journal articles. Articles found in trusted journals are one of the best places to find trustworthy information. For something to be published there, it has to go through a long process of peer review.

What is peer review?

When you do a project for school, sometimes you have to work in a group. Each member of the group should play an important role and do their share of the work. But another important benefit to working in a group is that you can check to make sure that what each person does is good enough quality to turn in. By checking each other’s work, you make sure that what you are turning in is of a better quality than it might be if you had to work alone.

The same is true in the world of scientists and other scholars. When someone thinks they’ve learned new information about a subject, it’s not automatically accepted by the rest of the world. Instead, a group of other scientists or scholars who are experts in that subject have to look at the work. This is peer review. So peer review is the evaluation (or checking) of a work of research by other researchers who are experts in that area of study.

Some of the questions that are asked about work during peer review may include the following: 

• Was the experiment or study done properly? 

• Was the experiment or study repeatable? 

• Were there enough subjects or is there enough evidence? 

• Was the work analyzed correctly? 

• Do the findings and conclusions make sense according to everything else we know about the subject? 

• Is it possible the findings could mean anything else? 

• Does it have the importance that the scientists think it might? 

During peer review, if any of these things don’t seem quite right, the work would not be accepted. If the work is presented in a way that makes it hard for others to do the same tests if they wanted to, it also would not be accepted. This means that there is a long list of ways to double check a researcher’s work. If the work isn’t considered to be of a high enough quality, it will not be considered new knowledge.

Why is peer review important?

We base so much of our lives on information. The ways we cook foods, the medicines that we use and trust, and the ways that we take care of our families are often based on information researchers have learned. If any researcher could just claim that whatever information they found was true, the world would be very different. Instead, we have the process of science that is supported by peer review, and this lets us discover wonderful and helpful things about our world. It’s important to have a strong, trustworthy process that lets us judge whether new information should be trusted or not.

What is the peer review process?

As a scientist, you spend a lot of time running your experiments, analyzing the data you gather, and interpreting what it all means. But none of that really matters unless you are able to share that information with others. This is why publishing your work, for others to see, is so important. 

When you submit a finished research paper to a journal, the long process of publication has only just started. First, the editor of the journal takes a look at what you’ve written. If they think the research was done correctly, and that it could be important work, they may decide to consider it and send it for peer review.

In peer review, two or more experts in the same field are asked to look at your paper to identify any issues in it. Based on how well the project was done and how well it’s written, they may reject it altogether, ask for revisions, or think it’s a great project and is close to being ready to publish. If the reviewers disagree, more experts may be asked to take a look. If revisions are needed, all of the authors and researchers have to fix the issues with the work and the paper before they send it back in.

Journals can be very picky about the subjects they will publish. So even if a project gets rejected from one journal, it may be accepted by another, as long as there aren’t big errors in the project. Researchers who want to get their work published may have to go to several journals to find a good fit for a specific article.

More peer review 

But even if an article is accepted for publication, peer review doesn’t end there. When an article is published, it is then sort of opened up to all the researchers in the world. If they read it and have an issue with the project that the reviewers missed, questions may be raised about the work. Dozens of researchers may chime in, pointing out any errors or issues with the experiment. Some journals may remove (or “retract”) an article, even after it has been published, if the initial reviewers missed some important issue with the project. 

More and more these days, researchers also have to allow access to the data the paper is based on. This allows other researchers to look to see if the data makes sense as well. It also shows that data was collected, and the researchers didn’t just make up some outcome of an experiment. 

Jumping from journal to journal

You read an article telling you that you can eat all the chocolate you want because the antioxidants in it make it healthy. Can you believe this claim? First, you’ll want to see what the article cites. “Ahhh, it links to a published paper,” you think… “this must mean it’s trustworthy!” But that’s not quite how it works. While published papers are often trustable, you may still need to look a little further to see which journal a research paper is published in. 

Journals are judged by the quality and importance of the work they publish. Some only publish work they know will be read and used by a huge number of other researchers. These journals have a higher “impact factor,” or grade. That grade can become important when you are trying to judge whether a journal is publishing accurate and unbiased articles or not. Some journals appear to be legitimate, but don’t follow a reliable peer review process. Their main purpose is to generate money from authors who pay to have their work published. These are known as predatory journals. If predatory journals even earn an impact factor, it’s usually very low (less than 1). 

So, if you see a paper that seems to have a huge health implication, like one saying that eating a pound of chocolate a day is okay, look a little deeper. It should be published in a journal with a much higher impact factor. If it has to do with human health, you may want to look for papers in journals that have an impact factor around 4 or higher. If the work is on other organisms, and does not have an impact on human health, lower numbers are okay… maybe 3 or 2… but even around 1 wouldn’t be too questionable for a very specific field.

You can search for the impact factor of specific journals here or with a Google search. There are also some sites to help you get a better idea of what makes a "good" impact factor.

Are there problems with peer review and can it fail?

Is peer review perfect? No, it has its flaws. Sometimes the process can fail and result in articles that are inaccurate. Sometimes the process may be biased. Authors sometimes race to publish and can push work through that isn’t quite ready. There are even some reports of researchers publishing with false data. But in those cases, errors are often caught later and articles with issues may be pulled from a journal. This is part of why it’s important to make data available with publications, and why researchers should be rewarded for replicating certain experiments that have already been done. 

Despite its issues, peer review provides a degree of verification for information published in most research journals. Without it, there is a greater risk of circulating false information. But if you know how to navigate it, it can help you be more sure that the information you decide to share is valid and helpful to your community.