Avoiding Methodological Biases in Meta-Analysis Use of Online Versus Offline Individual Participant Data (IPD) in Educational Psychology

作者：Kaufmann Esther; Reips Ulf Dietrich; Merki Katharina Maag

来源：Journal of Psychology, 2016, 224(3): 157-167.

DOI：10.1027/2151-2604/a000251

• meta-analysis
• ecological fallacy
• online versus offline
• Simpson's paradox
• replication

Individual participant data (IPD) meta-analysis is the gold standard of meta-analyses. This paper points out several advantages of IPD meta-analysis over classical meta-analysis, such as avoiding aggregation bias (e.g., ecological fallacy or Simpson's paradox) and shows how its two main disadvantages (time and cost) can be overcome through Internet-based research. Ideally, we recommend carrying out IPD meta-analyses that consider online versus offline data gathering processes and examine data quality. Through a comprehensive literature search, we investigated whether IPD meta-analyses published in the field of educational psychology already follow these recommendations; this was not the case. For this reason, the paper demonstrates characteristics of ideal meta-analysis on teachers' judgment accuracy and links it to recent meta-analyses on that topic. The recommendations are important for meta-analysis researchers and for readers and reviewers of meta-analyses. Our paper is also relevant to current discussions within the psychological community on study replication.

• 出版日期2016