Skip to Main Content

Systematized review methods for SBS 402

Information about the methods used in systematic reviews that can be adapted for SBS capstone projects.

Before you screen

You should have an RIS file from each of your three databases. 

Before uploading your RIS files to Rayyan, name the file in a way that makes it clear which database it's from and the date of your search. For example, my search in Academic Search Premier will be named ASP_Mar15.ris. 

We will do two rounds of screening: an initial screening, in which we screen the abstracts only, and a final screening, in which we screen the full text. 

Rayyan video

Rayyan: Creating a review and de-duplicating

Rayyan is the tool we'll be using to screen our articles. Start by signing up for a free account. Then create a new review. 

You will be prompted to upload your references, which are the RIS files you exported from Zotero or from each database. They should be named after the database and search date before you upload. 

The first step is to de-duplicate your articles, in case some of the same articles showed up in your searches from different databases. 

From the "possible duplicates" section in the left navigation, click "unresolved" to view the possible duplicates. Then click on one of the article titles to open it, and select "resolve duplicate." 

The two possible duplicates will appear in the lower box. If you can see that they are the same article, you can delete one. If not, select "not duplicate." 

Rayyan: Initial screening (abstracts)

For our initial screening, we will be reviewing article abstracts to see whether they meet our inclusion criteria. 

The "inclusion decisions" box allows you to view the "undecided" articles, which are the ones you need to screen. 

Because we are doing a systematized review, rather than a full systematic review, we are going to start by screening the most recent articles first, and continue until we get about 10-15 articles. 

Start by screening the sources from the most current year by selecting that year from the left navigation. 

Select one of the articles that comes up and read the abstract. There are buttons to include or exclude the article. This decision should be made based on your PICOS. Is the population correct? Is the intervention correct? Is the comparison correct? Is the outcome measured correct? Is the study type correct? 

If you exclude an article, you always need to record the reason in the reason box. You can choose from the reasons offered, or add your own. If the article seems completely irrelevant, you can put "irrelevant." Otherwise, choose "wrong population," "wrong study type," etc. You only have to choose one reason for exclusion, so as soon as you identify a reason to exclude a source, mark it and move on. 

If you are considering limiting your articles by geography (this may or may not be appropriate, depending on your research question), mark articles outside your potential geographic area of focus as "maybe." You can revisit these later, once you have decided whether to limit by geography.

At this stage, your goal is to screen until you've included about 10-15 articles. Important: Your decisions about which articles to include and exclude should be made by consistently applying your inclusion/exclusion criteria (PICOS). We are NOT making inclusion/exclusion decisions based on our subjective criteria for what we think sounds interesting, fits our topic, or will help us write our papers. 

If you've reached your 10-15 included articles, continue screening the rest of the articles published in that year so that your date range has a clean cut-off point. 

Revisit any "maybes" to determine whether or not to include them. If you don't have enough information from the abstract to determine whether it meets your criteria, include it. You will do a more thorough screening later. 

Clear all of the filters on the left side of the screen, then select "included" in the Inclusion Decisions box. Make sure the correct number of entries are displayed. 

Export the filtered list in RefMan format. 

Rayyan: Final screening (full text)

For our final screening, we will be reviewing the full text of the articles to see whether they meet our inclusion criteria. 

Click on the "all reviews" button in the top right part of the screen to get back to the page listing all of your reviews. 

Create a new review, and upload the file that you just downloaded from your initial screening of abstracts. You might name this review "full text review" so that you can easily distinguish it from the initial review. 

You will need to obtain the full text for all of the articles you will be screening in this round. If you are using Zotero, it may have already collected full text for a number of your sources. Keep your sources together in a folder for easy access. To obtain full text: 

  1. First, put the article title in OneSearch on the library website to see if we have full text in any of our databases. 
  2. If it doesn't come up in the initial results list, use the option "click here to expand your search" (above search results). 
  3. If it still doesn't come up, click "Still didn't find what you need? Click here to send a request." 
  4. You can also search Google to see whether the author has made the article freely available. 

You may want to upload the full text PDFs into your full text review in Rayyan so that they are easily accessible. Select an article and use the "upload PDF full-texts" button.

The PDF will show up as a link near the bottom of the abstract box. 

You don't have to read the entire article at this point; you are mostly looking to see whether the article meets your inclusion/exclusion criteria (PICOS). Re-read the abstract, then skip to the methods section, which should include the PICOS information. Create a note in Rayyan to document the PICOS for that article. 

Your notes will show up at the bottom of the abstract box: 

Use the include and exclude buttons to record your decisions, and make sure to mark reasons for exclusion. Through this round of screening, your criteria will hopefully narrow down your final group of articles to 5-8. 

PRISMA flow diagram

A PRISMA flow diagram will be included in the methods or results section of your paper and will document all of your steps that brought you to the final set of articles that you will analyze. You can see that you'll need to record the necessary numbers to populate this table. 

Here's an example of a PRISMA diagram: 


You can generate your diagram manually, or using a PRISMA flow diagram generator, such as this one