We present a data set containing 705 between-study heterogeneity estimates ^{2} as reported in 61 articles published in ^{2}-statistics, and publication bias. The data set is stored in the Open Science Framework repository (

The data were collected between July 2014 and September 2014.

Meta-analysis provides an important tool to synthesize evidence across different studies on the same topic. For the analysis the researcher typically needs to adopt either a fixed-effect model or a random-effects model. A fixed-effect meta-analysis assumes one true underlying effect size with variation in observed effect sizes arising solely because of sampling error. In contrast, a random-effects meta-analysis assumes a distribution of true effect sizes; consequently, observed differences in effect sizes arise both from sampling error and from true differences between effect sizes. The latter form of variation, or between-study heterogeneity, is the focus of this data set.

The random-effects model for

where the sampling error _{i}_{i}^{2}, i.e., _{i}^{2}). The estimate of the between-study heterogeneity, or ^{2}

It is unclear, however, what constitutes a “large” heterogeneity in the field of psychology. Therefore, it is important to obtain an overview of between-study heterogeneity estimates encountered in meta-analyses in psychology. Such an overview allows researchers to compare the ^{2} estimate in their meta-analysis to the general distribution of estimates and gauge the size of their between-study heterogeneity. In addition, when conducting a Bayesian meta-analysis, this overview can provide a basis for the construction of an informed prior distribution for between-study heterogeneity.

In the medical literature, several attempts have been made to present an overview of between-study heterogeneity estimates from a large number of meta-analyses and construct an informed prior distribution based on these estimates [^{2} estimates extracted from a large number of meta-analyses in the field of psychology. The data set presented here is the result of this literature review.

We extracted 705 between-study heterogeneity estimates ^{2} from meta-analyses reported in 61 articles published in

The selected time frame featured 255 articles reporting meta-analyses, but we included only studies that provided estimates of the between-study variance ^{2} or standard deviation

No materials were used, apart from a laptop to search for meta-analyses.

First, we searched for articles containing overviews of meta-analyses; however, these articles almost never reported the estimated ^{2} values. Instead, we focused on separate meta-analyses published in

“

Through the search engine Ebscohost, we searched each issue from 1990 up to and including 2013 by selecting “meta-analysis” as methodology or by searching on the keyword “meta-analysis”. If this gave no results, all titles in the issue were scanned for the keywords “meta-analysis”, “review”, or “synthesis”. The resulting 255 articles were scanned to determine whether a meta-analysis was performed and an estimate of ^{2} provided. This was the case in 61 articles. Since most articles included multiple analyses (such as separate meta-analyses for multiple outcome variables), we extracted several heterogeneity estimates per article, resulting in a final data set of 705 estimates. Note that different estimators for ^{2} exist (e.g., Hunter-Schmidt, DerSimonian-Laird, maximum likelihood), which can provide varying or even conflicting estimates of the between-study heterogeneity [^{2} was used. All meta-analyses were based on the random-effects model in equation 1. When articles provided the between-study standard deviation (variance), we computed the variance (standard deviation) so that the data set includes both ^{2} estimates for each meta-analysis. The data extraction and coding were performed manually by the first author.

The data were collected with the utmost care and conscientiousness. In addition, the Appendix provides an overview of all articles included in the data set so that the full data set can be easily checked.

Only secondary data were used to construct the data set.

Figure ^{2}; (5) the ^{2}-statistic (i.e., the percentage of total variation in effect sizes due to true between-study heterogeneity; computed based on the ^{2}-statistic, which does not depend on the type of effect size used. Note that the ^{2} statistic is fixed to zero when

Screenshot of the data file “Data 1990–2013 with tau values.xlsx”.

Histogram of the between-study standard deviation for mean difference effect sizes (Cohen’s

Histogram of the between study standard deviation for correlation effect sizes (Pearson’s

Histogram of the ^{2} statistic for all types of effect sizes.

The name of the file is “Data 1990–2013 with tau values.xlsx”.

Secondary data.

This is the third and final version of the data, in Excel, comma-separated values (CSV), and text format. In the first version, the type of effect size was missing for one article (Hartwig and Bond Jr., 2011). In the second version, we included the columns “Entry in reference”, “Analysis description”, “^{2}”, and “Test for publication bias” and removed a column in which the type of effect size was recoded. Moreover, we discovered coding errors for the column “Publication bias?”. Specifically, several articles which did not test for publication bias were coded as having no publication bias. This has been changed to NA for the articles: Else-quest et al. (2012); Hagger et al. (2010); van Zomeren et al. (2008); Postmes and Spears (1998); Ambady and Rosenthal (1992); and Raz and Raz (1990). In addition, we changed the recoded publication bias variable to remain NA if information regarding publication bias was not reported in the article, instead of coding this as 2.

The following R code can be used to read in the data and clean it:

Some heterogeneity estimates are missing for articles in which meta-analyses were conducted for multiple variables. In these cases, it sometimes happened that only one study was found for some reported variables and thus only one effect size estimate is available. These effect sizes have been included in the data set, but naturally there is no corresponding heterogeneity estimate available. An example is the article by DeNeve and Cooper (1998). In addition, heterogeneity estimates were sometimes missing despite the fact that multiple effect sizes were available. This occurred once in the article by DeNeve and Cooper (1998), who did not provide an explanation. It occurred multiple times in the article by Mathieu and Zajac (1990) when the between study heterogeneity was completely attributable to statistical artifacts.

A total of 90 heterogeneity estimates are equal to zero. When there exists no between-study variance, a fixed-effect meta-analysis is appropriate. However, some articles reported random-effects meta-analyses by default or reported both fixed-effect and random-effects meta-analyses, in which case the heterogeneity estimate can be zero.

Sara van Erp (Tilburg University) collected the data.

English.

CC0 1.0 Universal.

The data set is not under embargo.

The data set is published in the Open Science Framework repository, available at

The data set was originally published on 29/11/2015. The second version was published on 08/04/2017 and the final version was published on 17/07/2017.

The data set provides an overview of between-study heterogeneity estimates based on 61 meta-analyses in psychology. The main application of this data set is to facilitate the construction of an informed prior for the between-study heterogeneity in Bayesian meta-analysis, which has been done in [^{2} estimate to the distribution of estimates and assess the relative size of the between-study heterogeneity in their meta-analysis.

In addition to the ^{2} estimates of the between-study heterogeneity, the data set contains information on the number of studies in each meta-analysis, the

The additional files for this article can be found as follows:

List of included meta-analyses (all studies are published in

Data set containing 705 between-study heterogeneity estimates from meta-analyses published in

The authors have no competing interests to declare.