However, the rate of response from authors is lower than expected. In some areas, such as genetics, in which it was shown that it is possible to identify an individual using the aggregated statistics from a particular study [ 34 ], strict criteria are imposed for data sharing, and specialized permissions might be needed. For cases in which there is enough primary data of adequate quality for a quantitative summary, there is the option to carry out a meta-analysis.
The potential analyst must be warned that in many cases the data are reported in noncompatible forms, so one must be ready to perform various types of transformations.
Frequently, meta-analyses are based on fixed-effects or random-effects statistical models [ 20 ]. In addition, models based on combining ranks or p -values are also available and can be used in specific cases [ 41 — 44 ].
For more complex data, multivariate methods for meta-analysis have been proposed [ 45 , 46 ]. Additional statistical examinations involve sensitivity analyses, metaregressions, subgroup analyses, and calculation of heterogeneity metrics, such as Q or I 2 [ 20 ].
It is fundamental to assess and, if present, explain the possible sources of heterogeneity. Although random-effects models are suitable for cases of between-studies heterogeneity, the sources of between-studies variation should be identified, and their impact on effect size should be quantified using statistical tests, such as subgroup analyses or metaregression.
Publication bias is an important aspect to consider [ 47 ], since in many cases negative findings have less probability of being published.
There are several very user-friendly and freely available programs for carrying out meta-analyses [ 43 , 44 ], either within the framework of a statistical package such as Stata or R or as stand-alone applications. Some of these programs are web services or stand-alone software. In some cases, certain programs can present issues when they are run because of their dependency on other packages.
Following published guidelines for meta-analyses guarantees that the manuscript will describe the different steps and methods used, facilitating their transparency and replicability [ 15 ]. Data such as search and inclusion criteria, numbers of abstracts screened, and included studies are quite useful, in addition to details of meta-analytical strategies used. An assessment of quality of included studies is also useful [ 60 ]. A spreadsheet can be constructed in which every step in the selection criteria is recorded; this will be helpful to construct flow charts.
In this context, a flow diagram describing the progression between the different steps is quite useful and might enhance the quality of the meta-analysis [ 61 ].
Records will be also useful if, in the future, the meta-analysis needs to be updated. Stating the limitations of the analysis is also important [ 62 ]. A table with complete information about included studies such as author, year, details of included subjects, DOIs, or PubMed IDs, among others is quite useful in an article reporting a meta-analysis; it can be included in the main text of the manuscript or as a supplementary file.
Software used for carrying out meta-analyses and to generate key graphs, such as forest plots, should be referenced. Summary effect measures, such as a pooled odds ratios or the counts used to generate them, should be always reported, including confidence intervals. It is also possible to generate figures with information from multiple forest plots [ 63 ].
In the case of positive findings, plots from sensitivity analyses are quite informative. In more-complex analyses, it is advisable to include in the supplementary files the scripts used to generate the results [ 64 ].
The Discussion section is an important scientific component in a manuscript describing a meta-analysis, as the authors should discuss their current findings in the context of the available scientific literature and existing knowledge [ 65 ]. Authors can discuss possible reasons for the positive or negative results of their meta-analysis, provide an interpretation of findings based on available biological or epidemiological evidence, and comment on particular features of individual studies or experimental designs used [ 66 ].
As open science is becoming more important around the globe [ 68 , 69 ], adherence to published standards, in addition to the evolution of methods for different meta-analytical applications, will be even more important to carry out meta-analyses of high quality and impact. Introduction In the context of evidence-based medicine, meta-analyses provide novel and useful information [ 1 ], as they are at the top of the pyramid of evidence and consolidate previous evidence published in multiple previous reports [ 2 ].
Rule 1: Specify the topic and type of the meta-analysis Considering that a systematic review [ 10 ] is fundamental for a meta-analysis, you can use the Population, Intervention, Comparison, Outcome PICO model to formulate the research question. Rule 2: Follow available guidelines for different types of meta-analyses There are several available general guidelines. Rule 3: Establish inclusion criteria and define key variables You should establish in advance the inclusion such as type of study, language of publication, among others and exclusion such as minimal sample size, among others criteria.
Rule 4: Carry out a systematic search in different databases and extract key data You can carry out your systematic search in several bibliographic databases, such as PubMed, Embase, The Cochrane Central Register of Controlled Trials, Scopus, Web of Science, and Google Scholar [ 21 ]. Rule 5: Contact authors of primary articles to ask for missing data It is common that key data are not available in the main text or supplementary files of primary articles [ 31 ], leading to the need to contact the authors to ask for missing data.
Rule 6: Select the best statistical models for your question For cases in which there is enough primary data of adequate quality for a quantitative summary, there is the option to carry out a meta-analysis.
Rule 7: Use available software to carry metastatistics There are several very user-friendly and freely available programs for carrying out meta-analyses [ 43 , 44 ], either within the framework of a statistical package such as Stata or R or as stand-alone applications.
Rule 8: The records and study report must be complete and transparent Following published guidelines for meta-analyses guarantees that the manuscript will describe the different steps and methods used, facilitating their transparency and replicability [ 15 ]. Rule 9: Provide enough data in your manuscript A table with complete information about included studies such as author, year, details of included subjects, DOIs, or PubMed IDs, among others is quite useful in an article reporting a meta-analysis; it can be included in the main text of the manuscript or as a supplementary file.
Rule Provide context for your findings and suggest future directions The Discussion section is an important scientific component in a manuscript describing a meta-analysis, as the authors should discuss their current findings in the context of the available scientific literature and existing knowledge [ 65 ].
References 1. JAMA — Clin J Am Soc Nephrol 3: — Heart Lung Vessel 5: — Stat Med — Hedges LV The early history of meta-analysis. Res Synth Methods 6: — Glass GV Meta-analysis at middle age: a personal history. Pautasso M Ten simple rules for writing a literature review. PLoS Comput Biol 9: e BMJ f Syst Rev 2: 4. Onkologie — PLoS Med 6: e Human Genome Epidemiology Network.
PLoS Med 5: e Nat Rev Genet — J Neurosci Methods 92— Kavvoura FK, Ioannidis JP Methods for meta-analysis in genetic association studies: a review of their potential and pitfalls. Hum Genet 1— J Clin Epidemiol — Nucleic Acids Res D— Control Clin Trials 1— Stang A Critical evaluation of the Newcastle-Ottawa scale for the assessment of the quality of nonrandomized studies in meta-analyses.
Eur J Epidemiol — Ann Intern Med — Furthermore, despite the increasing guidelines for effectively conducting a systematic review, we found that basic steps often start from framing question, then identifying relevant work which consists of criteria development and search for articles, appraise the quality of included studies, summarize the evidence, and interpret the results [ 2 , 3 ].
However, those simple steps are not easy to be reached in reality. There are many troubles that a researcher could be struggled with which has no detailed indication. To solve those hindrances, we recommend a flow diagram Fig. Detailed flow diagram guideline for systematic review and meta-analysis steps. All the explained methods feature the standards followed internationally, with our compiled experience in the conduct of SR beside it, which we think proved some validity.
This is a SR under conduct by a couple of researchers teaming in a research group, moreover, as the outbreak of Ebola which took place — in Africa resulted in a significant mortality and morbidity. Furthermore, since there are many published and ongoing trials assessing the safety of Ebola vaccines, we thought this would provide a great opportunity to tackle this hotly debated issue.
Moreover, Ebola started to fire again and new fatal outbreak appeared in the Democratic Republic of Congo since August , which caused infection to more than people according to the World Health Organization, and people have been killed till now.
Hence, it is considered the second worst Ebola outbreak, after the first one in West Africa in , which infected more than 26, and killed about 11, people along outbreak course.
Therefore, a clear, logical, and well-defined research question should be formulated. When we apply this to our assumed research topic, being of qualitative nature, the use of SPIDER approach is more valid. PICO is usually used for systematic review and meta-analysis of clinical trial study.
For the observational study without intervention or comparator , in many tropical and epidemiological questions, it is usually enough to use P Patient and O outcome only to formulate a research question.
We must indicate clearly the population P , then intervention I or exposure. Next, it is necessary to compare C the indicated intervention with other interventions, i. Finally, we need to clarify which are our relevant outcomes. To facilitate comprehension, we choose the Ebola virus disease EVD as an example.
Currently, the vaccine for EVD is being developed and under phase I, II, and III clinical trials; we want to know whether this vaccine is safe and can induce sufficient immunogenicity to the subjects.
P: healthy subjects human , I: vaccination, C: placebo, O: safety or adverse effects. We recommend a preliminary search to identify relevant articles, ensure the validity of the proposed idea, avoid duplication of previously addressed questions, and assure that we have enough articles for conducting its analysis. Moreover, themes should focus on relevant and important health-care issues, consider global needs and values, reflect the current science, and be consistent with the adopted review methods.
Gaining familiarity with a deep understanding of the study field through relevant videos and discussions is of paramount importance for better retrieval of results. If we ignore this step, our study could be canceled whenever we find out a similar study published before.
This means we are wasting our time to deal with a problem that has been tackled for a long time. While doing this step, we identify a systematic review and meta-analysis of determinant factors influencing antibody response from vaccination of Ebola vaccine in non-human primate and human [ 7 ], which is a relevant paper to read to get a deeper insight and identify gaps for better formulation of our research question or purpose.
We can still conduct systematic review and meta-analysis of Ebola vaccine because we evaluate safety as a different outcome and different population only human.
Eligibility criteria are based on the PICO approach, study design, and date. Exclusion criteria mostly are unrelated, duplicated, unavailable full texts, or abstract-only papers. These exclusions should be stated in advance to refrain the researcher from bias. The inclusion criteria would be articles with the target patients, investigated interventions, or the comparison between two studied interventions. Briefly, it would be articles which contain information answering our research question.
But the most important is that it should be clear and sufficient information, including positive or negative, to answer the question. For the topic we have chosen, we can make inclusion criteria: 1 any clinical trial evaluating the safety of Ebola vaccine and 2 no restriction regarding country, patient age, race, gender, publication language, and date.
Exclusion criteria are as follows: 1 study of Ebola vaccine in non-human subjects or in vitro studies; 2 study with data not reliably extracted, duplicate, or overlapping data; 3 abstract-only papers as preceding papers, conference, editorial, and author response theses and books; 4 articles without available full text available; and 5 case reports, case series, and systematic review studies.
A standard search strategy is used in PubMed, then later it is modified according to each specific database to get the best relevant results. The basic search strategy is built based on the research question formulation i. Search strategies are constructed to include free-text terms e.
Additionally, we advise not to use terms for the Outcomes as their inclusion might hinder the database being searched to retrieve eligible studies because the used outcome is not mentioned obviously in the articles.
The improvement of the search term is made while doing a trial search and looking for another relevant term within each concept from retrieved papers. Because the study for this topic is limited, we do not include outcome term safety and immunogenicity in the search term to capture more studies.
Researchers should consider to choose relevant database according to the research topic. Some databases do not support the use of Boolean or quotation; otherwise, there are some databases that have special searching way.
Therefore, we need to modify the initial search terms for each database to get appreciated results; therefore, manipulation guides for each online database searches are presented in Additional file 5 : Table S2.
The detailed search strategy for each database is found in Additional file 5 : Table S3. The search term that we created in PubMed needs customization based on a specific characteristic of the database. An example for Google Scholar advanced search for our topic is as follows:.
Finally, all records are collected into one Endnote library in order to delete duplicates and then to it export into an excel sheet. Using remove duplicating function with two options is mandatory.
All references which have 1 the same title and author, and published in the same year, and 2 the same title and author, and published in the same journal, would be deleted. References remaining after this step should be exported to an excel file with essential information for screening. Protocol registration at an early stage guarantees transparency in the research process and protects from duplication problems.
It is recommended that researchers send it to the principal investigator PI to revise it, then upload it to registry sites. Decisions to select retrieved articles for further assessment are based on eligibility criteria, to minimize the chance of including non-relevant articles. According to the Cochrane guidance, two reviewers are a must to do this step, but as for beginners and junior researchers, this might be tiresome; thus, we propose based on our experience that at least three reviewers should work independently to reduce the chance of error, particularly in teams with a large number of authors to add more scrutiny and ensure proper conduct.
Mostly, the quality with three reviewers would be better than two, as two only would have different opinions from each other, so they cannot decide, while the third opinion is crucial. And here are some examples of systematic reviews which we conducted following the same strategy by a different group of researchers in our research group and published successfully, and they feature relevant ideas to tropical medicine and disease [ 9 , 10 , 11 ].
In this step, duplications will be removed manually whenever the reviewers find them out. When there is a doubt about an article decision, the team should be inclusive rather than exclusive, until the main leader or PI makes a decision after discussion and consensus.
All excluded records should be given exclusion reasons. Many search engines provide links for free to access full-text articles. In case not found, we can search in some research websites as ResearchGate, which offer an option of direct full-text request from authors.
Additionally, exploring archives of wanted journals, or contacting PI to purchase it if available. Similarly, 2—3 reviewers work independently to decide about included full texts according to eligibility criteria, with reporting exclusion reasons of articles.
In case any disagreement has occurred, the final decision has to be made by discussion. One has to exhaust all possibilities to reduce bias by performing an explicit hand-searching for retrieval of reports that may have been dropped from first search [ 12 ]. Each of the abovementioned methods can be performed by 2—3 independent reviewers, and all the possible relevant article must undergo further scrutiny against the inclusion criteria, after following the same records yielded from electronic databases, i.
Similarly, the number of included articles has to be stated before addition to the overall included records. This step entitles data collection from included full-texts in a structured extraction excel sheet, which is previously pilot-tested for extraction using some random studies.
We recommend extracting both adjusted and non-adjusted data because it gives the most allowed confounding factor to be used in the analysis by pooling them later [ 13 ].
The process of extraction should be executed by 2—3 independent reviewers. Mostly, the sheet is classified into the study and patient characteristics, outcomes, and quality assessment QA tool. Data presented in graphs should be extracted by software tools such as Web plot digitizer [ 14 ].
Most of the equations that can be used in extraction prior to analysis and estimation of standard deviation SD from other variables is found inside Additional file 5 : File S2 with their references as Hozo et al. We recommend that 2—3 reviewers independently assess the quality of the studies and add to the data extraction form before the inclusion into the analysis to reduce the risk of bias.
In the NIH tool for observational studies—cohort and cross-sectional—as in this EBOLA case, to evaluate the risk of bias, reviewers should rate each of the 14 items into dichotomous variables: yes, no, or not applicable. An overall score is calculated by adding all the items scores as yes equals one, while no and NA equals zero.
A score will be given for every paper to classify them as poor, fair, or good conducted studies, where a score from 0—5 was considered poor, 6—9 as fair, and 10—14 as good. In the EBOLA case example above, authors can extract the following information: name of authors, country of patients, year of publication, study design case report, cohort study, or clinical trial or RCT , sample size, the infected point of time after EBOLA infection, follow-up interval after vaccination time, efficacy, safety, adverse effects after vaccinations, and QA sheet Additional file 6 : Data S1.
Due to the expected human error and bias, we recommend a data checking step, in which every included article is compared with its counterpart in an extraction sheet by evidence photos, to detect mistakes in data. We advise assigning articles to 2—3 independent reviewers, ideally not the ones who performed the extraction of those articles. When resources are limited, each reviewer is assigned a different article than the one he extracted in the previous stage.
Investigators use different methods for combining and summarizing findings of included studies. Before analysis, there is an important step called cleaning of data in the extraction sheet, where the analyst organizes extraction sheet data in a form that can be read by analytical software. The analysis consists of 2 types namely qualitative and quantitative analysis. Qualitative analysis mostly describes data in SR studies, while quantitative analysis consists of two main types: MA and network meta-analysis NMA.
Subgroup, sensitivity, cumulative analyses, and meta-regression are appropriate for testing whether the results are consistent or not and investigating the effect of certain confounders on the outcome and finding the best predictors. Publication bias should be assessed to investigate the presence of missing studies which can affect the summary. Other Ebola vaccines were not meta-analyzed because of the limited number of studies instead, it will be included for narrative review.
The imaginary data for vaccine safety meta-analysis can be accessed in Additional file 7 : Data S2. To do the meta-analysis, we can use free software, such as RevMan [ 22 ] or R package meta [ 23 ].
In this example, we will use the R package meta. The R codes and its guidance for meta-analysis done can be found in Additional file 5 : File S3. For the analysis, we assume that the study is heterogenous in nature; therefore, we choose a random effect model. We did an analysis on the safety of Ebola vaccine A. From the data table, we can see some adverse events occurring after intramuscular injection of vaccine A to the subject of the study.
Suppose that we include six studies that fulfill our inclusion criteria. We can do a meta-analysis for each of the adverse events extracted from the studies, for example, arthralgia, from the results of random effect meta-analysis using the R meta package. From the results shown in Additional file 3 : Figure S3, we can see that the odds ratio OR of arthralgia is 1.
In the meta-analysis, we can also visualize the results in a forest plot. It is shown in Fig. The green box represents the effect size in this case, OR of each study. The bigger the box means the study weighted more i.
The blue diamond shape represents the pooled OR of the six studies. To evaluate publication bias related to the meta-analysis of adverse events of arthralgia, we can use the metabias function from the R meta package Additional file 4 : Figure S4 and visualization using a funnel plot. The results of publication bias are demonstrated in Fig. We see that the p value associated with this test is 0. We can confirm it by looking at the funnel plot. Looking at the funnel plot, the number of studies at the left and right side of the funnel plot is the same; therefore, the plot is symmetry, indicating no publication bias detected.
Sensitivity analysis is a procedure used to discover how different values of an independent variable will influence the significance of a particular dependent variable by removing one study from MA. It is only performed when there is a significant association, so if the p value of MA done is 0. For more assurance on the quality of results, the analyzed data should be rechecked from full-text data by evidence photos, to allow an obvious check for the PI of the study.
Writing based on four scientific sections: introduction, methods, results, and discussion, mostly with a conclusion. Performing a characteristic table for study and patient characteristics is a mandatory step which can be found as a template in Additional file 5 : Table S3. After finishing the manuscript writing, characteristics table, and PRISMA flow diagram, the team should send it to the PI to revise it well and reply to his comments and, finally, choose a suitable journal for the manuscript which fits with considerable impact factor and fitting field.
We need to pay attention by reading the author guidelines of journals before submitting the manuscript. The role of evidence-based medicine in biomedical research is rapidly growing. Having the basic steps for conduction of MA, there are many advanced steps that are applied for certain specific purposes. One of these steps is meta-regression which is performed to investigate the association of any confounder and the results of the MA.
In NMA, we investigate the difference between several comparisons when there were not enough data to enable standard meta-analysis. It uses both direct and indirect comparisons to conclude what is the best between the competitors.
0コメント