For most evidence syntheses, the bulk of your time is spent in screening your sources for relevancy according to the inclusion and exclusion criteria specified in the research protocol, followed by extracting and synthesizing the data within them. Many methods, such as systematic reviews, include appraising the quality of sources at this stage.
Depending on the amount of sources you find, it might make sense to use a simple program like Excel or find a specialized software platform for more complex data management and synthesis. See the Software Tools section in this guide for suggestions and resources.
The purpose of article screening is to remove sources that were retrieved during the search stage, but are not actually related to your topic. This stage is typically performed in two steps:
First screening: Title & abstract
Use your inclusion and exclusion criteria to first screen the title and abstracts of your sources and determine whether they are relevant to your research question.
In most cases, it's recommended that two independent reviewers screen all studies. Disagreement can be resolved by consensus or by consulting a third-party expert.
Second screening: Full-text
Once titles and abstracts have been screened, the full-text of the source must be retrieved in order to determine whether the study fits the eligibility criteria of your synthesis.
This is a more rigourous, in-depth step in which the articles that were included in the initial stage of screening are read in full-text. For most methods, you must record the reason for excluding sources so it can be reported in the write-up.
Below are some resources on article screening. You may also find it useful to consult relevant Reporting and conducting guidelines.
Evidence synthesis methods typically involve a strategic, structured approach to extracting data from sources that meet your inclusion and exclusion criteria. The exact process and tools you need will depend largely on your particular topic and the nature and amount of sources you need to synthesize.
In most cases, you will want to present sources in a table alongside written analysis. Reporting and conducting guidelines can provide helpful guidance on how to perform and report the data.
Also referred to as a risk of bias assessment, appraisal refers to evaluating the strength and quality of the source by considering, for example, an individual study’s design and limitations. There are many articles and checklists that can be used to guide appraisal, which are often specific to the subject area (e.g. Assessing the Risk of Bias in Systematic Reviews of Health Care Interventions and ReLIANT: Reader’s guide to the Literature on Interventions Addressing the Need for education and Training) or study design (e.g. Critical appraisal checklist for a case study). Many researchers use forms to ensure consistency among team members.
Synthesis is the process of bringing together data from a set of included studies in order to reach conclusions about a body of evidence. Again, the approach you should take will often depend on the topic and its disciplinary standards. For qualitative research, you might synthesize the data according to theme. For quantitative research, you might consider applying statistical analysis, which is referred to as meta-analysis.