Zhai, Chunjuan; Guyatt, Gordon

A systematic review synthesizes evidence addressing a structured clinical question using systematic and explicit methods to search, identify, screen, critically appraise, and extract and analyze data from relevant studies.^[1] A meta-analysis, on the other hand, is a statistical method that combines the quantitative results of available primary studies addressing the same question to generate a pooled summary estimate and confidence interval (CI).^[2] Meta-analyses can increase statistical power, enhance precision, and answer questions that single trials are underpowered to address. Although there are systematic reviews where meta-analyses are not appropriate, when they are well-conducted and reported, systematic reviews combined with meta-analyses provide valuable information for clinicians, researchers, and policymakers.

To better understand fixed-effect and random-effects models, the following analogy is drawn. Fifty teachers are enrolled in a study of a new math curriculum. For each teacher, the classes are randomized so that half the classes receive the old curriculum and half receive the new curriculum. The researcher then evaluates the effectiveness of the curricula in optimizing test scores.

What is this research trying to address? There are two questions with two underlying assumptions.

Question 1. Among these 50 teachers and no others, what is the impact of these two curricula on student examination scores? (Assumption 1: The effect of the new vs. the old curriculum is the same in all teachers.)

Question 2. Among all teachers who might ever teach this math course, of whom these 50 teachers are a random sample, what is the impact of these two curricula on student examination scores? (Assumption 2: The effect of the new vs. the old curriculum differs among teachers. For example, some teachers are better suited to the new curriculum than others.)

The differences between these two scenarios: In terms of the questions: Are we interested in the effect of the curricula in these 50 teachers or the effect in all teachers? In terms of assumptions: The relative effect of the old vs. the new curricula is the same or different across teachers.

Substitute “studies” for teachers and “therapies” for curricula and you get the questions and assumptions for fixed-effect (question 1) and random-effects (question 2) models.