An umbrella review looks, at first, like the easiest form of evidence synthesis, because the hard work of screening thousands of primary studies has already been done by the systematic reviews you are summarizing. It is not the easiest. It carries a methodological problem that no other review type faces in the same form: the same primary study can appear in several of the systematic reviews you include, and if you count its result more than once, you inflate the evidence and produce a synthesis that is quantitatively wrong. Managing that overlap, and reporting the whole review to a standard built specifically for overviews, is what separates a defensible umbrella review from one that double-counts its way to a false conclusion. Most guidance online never mentions overlap at all, and much of it misstates the reporting standard. Our systematic review writing service conducts umbrella reviews to the current standard. Still, any researcher attempting one needs to understand the two things that decide whether it holds: PRIOR and overlap.
This guide defines what an umbrella review is, distinguishes it from a systematic and a scoping review, sets out the PRIOR reporting standard and what makes it distinctive, and gives you the exact method, formula, and interpretation bands for measuring and managing primary-study overlap.
Quick Answer:
An umbrella review, also called an overview of reviews, synthesizes existing systematic reviews rather than primary studies: its unit of analysis is the review, not the study. It is appropriate when two or more systematic reviews already exist on related questions. Its central methodological threat is overlap: the same primary study appearing in multiple included reviews, which double-counts evidence and falsely narrows confidence intervals. Overlap is measured with the corrected covered area (CCA), calculated from a citation matrix, and interpreted against set bands (0 to 5 percent slight, 6 to 10 percent moderate, 11 to 15 percent high, above 15 percent very high). Umbrella reviews are reported to the PRIOR statement (Gates et al., BMJ 2022), which has 27 main items and 19 sub-items, and are not reported to PRISMA 2020, which was not designed for overviews.
What an Umbrella Review Actually Is
An umbrella review is a review of systematic reviews. Where a systematic review synthesizes primary studies to answer a focused question, an umbrella review sits one level up: it collects, appraises, and summarizes the existing systematic reviews on a topic, and its unit of analysis is the systematic review itself. The terminology varies by tradition. JBI, whose methodology group formalized the approach, calls it an umbrella review; Cochrane calls it an overview of reviews. The two terms are broadly synonymous, though they arise from different methodological manuals, and it is worth naming which tradition you are following in your own methods.
The design is appropriate in a specific circumstance: when two or more systematic reviews already exist on related questions, typically a broad condition addressed by several competing interventions, or a single intervention whose effects across many outcomes have each been reviewed separately. In that situation, a decision-maker does not need another primary synthesis; they need the existing syntheses brought together, compared, and appraised. That is the umbrella review's job. It is not a shortcut around systematic review methodology, and it is not appropriate simply because primary studies are numerous. It is appropriate when the reviews themselves have become the evidence base that needs summarizing.
How an Umbrella Review Differs From Systematic and Scoping Reviews
The three review types are frequently conflated, and getting the distinction right is the first thing an examiner or editor checks, because it determines every methodological choice that follows.
A systematic review answers a focused question by synthesizing primary studies, including a critical appraisal of those studies, and may pool them quantitatively in a meta-analysis. Its unit of analysis is the primary study. A scoping review maps the breadth and nature of the literature to clarify concepts and identify gaps; it typically does not critically appraise study quality, because its purpose is coverage rather than certainty. Its unit of analysis is also the primary study, but its aim is different. An umbrella review, distinct from both, takes systematic reviews as its unit of analysis, appraises those reviews, and summarizes what they collectively show. Choosing the wrong one of these three is one of the most consequential early errors in evidence synthesis, which our guide on choosing the right evidence synthesis works through in full.
Table 1: Umbrella Review vs Systematic Review vs Scoping Review
Feature | Umbrella Review | Systematic Review | Scoping Review |
|---|---|---|---|
Unit of analysis | The systematic review | The primary study | The primary study |
Question | Summarize what existing reviews show | Answer a focused question | Map breadth and identify gaps |
Critical appraisal | Of the included reviews (AMSTAR-2) | Of the primary studies (RoB tools) | Typically none |
Reporting standard | PRIOR (2022) | PRISMA 2020 | PRISMA-ScR |
Distinctive threat | Primary-study overlap (double-counting) | Incomplete search; bias in included studies | Unfocused scope |
The PRIOR Reporting Standard
Until recently, umbrella reviews were reported using PRISMA, which was designed for systematic reviews of primary studies and does not fit the overview context. That gap was closed in 2022 with the publication of PRIOR, the Preferred Reporting Items for Overviews of Reviews (Gates M, Gates A, Pieper D, Fernandes RM, Tricco AC, Moher D, et al. "Reporting guideline for overviews of reviews of healthcare interventions: development of the PRIOR statement." BMJ 2022;378:e070849). PRIOR was developed through an EQUATOR-registered process involving a large expert Delphi panel, a consensus meeting, and pilot testing, and it is the reporting standard that an umbrella review should now follow.
Two facts about PRIOR are worth stating precisely, because competitors routinely get them wrong. First, the checklist contains 27 main items with 19 sub-items across all stages of an overview. It is not a "56-item" checklist, a figure that circulates online and is incorrect. Second, PRIOR was deliberately not built as a simple PRISMA extension, because overviews present methodological challenges that PRISMA does not address. That distinction is the whole point of the standard, and it shows up in the specific items PRIOR adds.
What makes PRIOR distinctive is a set of items that exist because the unit of analysis is the review rather than the study. It asks authors to define what they mean by "systematic review" in their inclusion criteria (a novel sub-item), because "systematic review" is used loosely in the literature, and an overview must state which reviews qualify. It requires the identification and management of primary-study overlap at study selection, the description of methods to identify and quantify overlap for each outcome, and the reporting of the extent of overlap across included reviews. It requires planning for and discussing discordance, what to do when the included reviews disagree with one another. It requires appraising the methodological quality of the included systematic reviews themselves, and reporting the risk of bias of the primary studies as carried up from those reviews. And it requires assessing the certainty of the evidence, drawn from the included reviews or reassessed. Several of these items, overlap, discordance, and appraising the included reviews, have no counterpart in PRISMA 2020, because they are problems unique to synthesizing reviews rather than studies.
The Overlap Problem: Why Double-Counting Corrupts an Umbrella Review
The defining methodological threat in an umbrella review is overlap, and understanding why it is dangerous is the key to conducting one correctly. When you include several systematic reviews on the same topic, those reviews will often share primary studies. A single influential randomized trial might appear in five of the eight reviews you include. If, in summarizing or re-pooling the evidence, you count that trial's participants and events once for each review that contains it, you treat one study as if it were five, inflating the effective sample size and event counts. The consequence is a summary estimate whose confidence interval is falsely narrow and whose apparent precision is an artifact of counting the same people repeatedly. The synthesis looks more certain than the evidence warrants, and that false certainty is exactly what a decision-maker relies on and an appraiser catches.
The empirical record shows how routinely this is mishandled. When Pieper and colleagues examined how overviews dealt with overlap, they found that only about half mentioned it at all. The median corrected covered area across the overviews they studied was low, meaning most authors were not measuring the thing most likely to distort their results (Pieper D, Antoine SL, Mathes T, Neugebauer EA, Eikermann M. "Systematic review finds overlapping reviews were not mentioned in every other overview." Journal of Clinical Epidemiology 2014;67(4):368-375). The problem is not that overlap is rare; it is that it is common and frequently ignored.
Measuring Overlap: The Citation Matrix and the Corrected Covered Area
Overlap is not something you estimate impressionistically. It is quantified, and the standard method is the corrected covered area, or CCA, built from a citation matrix.
The citation matrix is the foundation. Build a table in which each row is a unique primary study and each column is one of the systematic reviews you have included. In each cell, enter a 1 if that primary study is included in that review and a 0 if it is not. The matrix makes the overlap visible: a primary study with five 1s across its row appears in five of your reviews. This same matrix is what PRIOR asks you to present and what your CCA calculation draws on.
From the matrix, the corrected covered area is calculated with a defined formula:
CCA = (N − r) / (r × c − r)
Here, N is the total number of times primary studies appear across all reviews, that is, the total count of 1s in the matrix, including every repeat; r is the number of unique primary studies, the number of rows; and c is the number of systematic reviews, the number of columns. The denominator, r times c minus r, is equivalent to r times (c minus 1). The CCA corrects the simpler "covered area" measure by accounting for the number of reviews, which is why it is the reported standard rather than a raw overlap count.
Interpreting the CCA and the Trap of the Overall Figure
A CCA is a percentage, and it is interpreted against established bands. These bands are what let you state, defensibly, how much overlap your umbrella review carries.
Table 2: Corrected Covered Area (CCA) Interpretation Bands (Pieper et al., 2014)
CCA Value | Degree of Overlap | What It Signals |
|---|---|---|
0 to 5% | Slight | Included reviews share a few primary studies |
6 to 10% | Moderate | Some shared studies; manage in synthesis |
11 to 15% | High | Substantial sharing; double-counting risk is real |
Above 15% | Very high | Reviews largely draw on the same studies |
The bands, from Pieper and colleagues, are: 0 to 5 percent is slight overlap, 6 to 10 percent is moderate overlap, 11 to 15 percent is high overlap, and above 15 percent is very high overlap. Reporting your CCA and its band is a PRIOR expectation and tells your reader how much the evidence base is shared across your included reviews.
There is a trap here that separates a competent umbrella review from an excellent one, and most guidance omits it entirely. A single overall CCA, calculated across the whole matrix, can hide concentrated pockets of very high overlap between particular pairs of reviews. Two reviews might share almost all their studies, while the overall figure, diluted by many non-overlapping studies elsewhere, looks moderate. Hennessy and Johnson set out guidance for handling this: calculate the overall CCA, then inspect the matrix for pairs of reviews with near-complete overlap, and calculate CCA for specific outcomes by subsetting the matrix to the reviews contributing to each outcome (Hennessy EA, Johnson BT. "Examining overlap of included studies in meta-reviews: guidance for using the corrected covered area index." Research Synthesis Methods 2020;11(1):134-145). Reporting overlap at the outcome level, not just overall, is what demonstrates you have actually controlled for double-counting rather than merely acknowledged it. Purpose-built tools now exist to compute these figures, including the ccaR package, so the calculation itself is no longer the barrier; understanding what to calculate is.
Appraising the Included Reviews and Rating Certainty
Because an umbrella review's inputs are systematic reviews, you must appraise the quality of those reviews, and this is a PRIOR requirement, not an optional extra. The standard instrument is AMSTAR-2, which rates the methodological quality of each included review and tells your reader how much confidence to place in each one; a review you include that is itself rated critically low should be weighted accordingly in your synthesis. Our guide on the AMSTAR-2 critical domains explains exactly how that appraisal works and why so many reviews score poorly. Alongside quality appraisal, you assess the certainty of the evidence for each outcome using GRADE, either carrying the ratings up from the included reviews or reassessing them yourself, so that your umbrella review reports not just what the reviews found but how much confidence the underlying evidence supports.
Planning an umbrella review and unsure how to handle overlap? |
|---|
Send us your topic and your candidate reviews. A methodologist will build your citation matrix, calculate overall and outcome-level corrected covered area, appraise each included review with AMSTAR-2, and structure the whole overview to the PRIOR checklist. Request umbrella review support and receive an itemized quote within 2 to 4 business hours, no obligation. |
Handling Discordance When Reviews Disagree
A problem unique to umbrella reviews, and one PRIOR explicitly requires you to address, is discordance: what to do when the systematic reviews you include reach different or contradictory conclusions on the same question. This is common because reviews differ in their eligibility criteria, their search dates, their included studies, and their analytical choices, and those differences produce different answers. An umbrella review that simply reports the divergent conclusions side by side without examining why they differ has not done its job. PRIOR asks you to plan, in your methods, how you will manage discordant data, and to summarize and explain the discrepancies in your discussion. In practice, this means tracing disagreement back to its source, differences in included studies (which your overlap matrix helps reveal), in the timing of searches, or in synthesis methods, and telling your reader which review's conclusion is better supported and why. Handling discordance well is one of the clearest signals of a rigorous umbrella review.
Conducting and Reporting an Umbrella Review, Step by Step
Bringing the pieces together, a defensible umbrella review proceeds in a defined order. Confirm first that the design is appropriate, that two or more systematic reviews exist, and that the question is about summarizing reviews rather than primary studies, and register your protocol. Define what counts as a "systematic review" for your inclusion criteria, following PRIOR, and decide whether you will permit any supplemental primary studies. Search for and select the systematic reviews, managing overlap considerations from the selection stage. Appraise each included review with AMSTAR-2 and record the risk of bias of the primary studies as reported in those reviews. Build the citation matrix, calculate the overall corrected covered area and the pairwise or outcome-level CCA, interpret them against the bands, and present both the matrix and the figures. Synthesise so that each primary study contributes only once to any pooled result, manage discordance explicitly, and assess certainty with GRADE. Report the whole overview against PRIOR's 27 items and 19 sub-items. Where a quantitative element is involved, sound pooling still matters, and our guide on how to do a meta-analysis and our meta-analysis service cover the statistical side; the appraisal underpinning it all is detailed in our guide on critically appraising studies.
Frequently Asked Questions
What is an umbrella review?
An umbrella review, also called an overview of reviews, is a review of existing systematic reviews rather than of primary studies: its unit of analysis is the systematic review. It is appropriate when two or more systematic reviews already exist on related questions, typically a condition addressed by several interventions or an intervention reviewed across many outcomes. It collects, appraises, and summarizes those reviews rather than synthesizing primary studies directly.
How is an umbrella review different from a systematic review?
A systematic review synthesizes primary studies to answer a focused question and appraises those studies, with the primary study as its unit of analysis. An umbrella review sits one level up: it synthesizes existing systematic reviews, appraises those reviews (usually with AMSTAR-2), and summarizes what they collectively show. You conduct an umbrella review when the systematic reviews themselves have become the evidence base that needs bringing together, not when you simply have many primary studies.
What is the PRIOR statement?
PRIOR (Preferred Reporting Items for Overviews of Reviews) is the reporting guideline for overviews of reviews of healthcare interventions, published by Gates and colleagues in the BMJ in 2022. It contains 27 main items with 19 sub-items and was designed specifically for overviews, not as a simple PRISMA extension, because overviews raise methodological issues such as primary-study overlap, discordance between reviews, and appraisal of the included reviews that PRISMA 2020 does not address.
How many items does PRIOR have?
PRIOR has 27 main items with 19 sub-items across all stages of an overview. The figure of "56 items" that circulates online is incorrect. The distinctive items, relative to PRISMA 2020, cover defining what a systematic review is for inclusion, identifying and quantifying primary-study overlap, managing and discussing discordance between reviews, appraising the methodological quality of the included reviews, and assessing certainty of evidence.
What is the corrected covered area (CCA)?
The corrected covered area is the standard measure of primary-study overlap in an umbrella review. It is calculated from a citation matrix (rows are unique primary studies, columns are the included reviews, cells are 1 or 0) using the formula CCA = (N − r) / (r × c − r), where N is the total number of study occurrences across all reviews, r is the number of unique primary studies, and c is the number of reviews. It is interpreted against set bands: 0 to 5 percent slight, 6 to 10 percent moderate, 11 to 15 percent high, and above 15 percent very high.
Why does overlap matter in an umbrella review?
Because the same primary study often appears in several of the reviews you include, and counting its result once per review double-counts the evidence, that inflates the effective sample size and event counts, falsely narrows confidence intervals, and makes the synthesis look more certain than the evidence supports. Measuring overlap with the CCA and ensuring each primary study contributes only once to any pooled result is what prevents this distortion, and reporting it is a PRIOR requirement.
Should I report one overall CCA or several?
Both. Report the overall CCA for the whole matrix, but also examine pairwise and outcome-level overlap, because a single overall figure can hide concentrated pockets of very high overlap between particular reviews. Two reviews may share almost all their studies, while the overall CCA, diluted by non-overlapping studies elsewhere, looks moderate. Reporting overlap at the outcome level, as Hennessy and Johnson recommend, demonstrates you have actually controlled for double-counting rather than merely acknowledged it.
You May Also Find Useful
Getting the Two Things That Matter Right
An umbrella review stands or falls on two things: whether you measured and managed the overlap of primary studies across your included reviews, and whether you reported the whole overview to the standard built for it. Build the citation matrix, calculate the corrected covered area overall and by outcome, appraise every included review, handle discordance explicitly, and report to PRIOR's 27 items and 19 sub-items. Do that, and your umbrella review summarizes the evidence accurately instead of double-counting its way to false precision, which is the difference between a synthesis decision-makers can use and one an appraiser sets aside.
If you want a methodologist to build your citation matrix, calculate your corrected covered area, and structure your overview to PRIOR, send us your topic and candidate reviews. You will have an itemized quote within 2 to 4 business hours, with no obligation.
