A scalable nonparametric specification testing for massive data

作者：Zhao, Yanyan; Zou, Changliang; Wang, Zhaojun^*

来源：Journal of Statistical Planning and Inference, 2019, 200: 161-175.

DOI：10.1016/j.jspi.2018.09.012

• Adaptive test
• Asymptotic normality
• Lack-of-fit test
• Rate-optimal
• Sample-splitting method

Lack-of-fit checking for parametric models is essential in reducing misspecification. However, for massive data sets which are increasingly prevalent, classical tests become prohibitively costly in computation and their feasibility is questionable even with modern parallel computing platforms. Building on the divide and conquer strategy, we propose a new nonparametric testing method, that is fast to compute and easy to implement with only one tuning parameter determined by a given time budget. Under mild conditions, we show that the proposed test statistic is asymptotically equivalent to that based on the whole data. Benefiting from using the sample-splitting idea for choosing the smoothing parameter, the proposed test is able to retain the type-I error rate pretty well with asymptotic distributions and achieves adaptive rate-optimal detection properties. Its advantage relative to existing methods is also demonstrated in numerical simulations and a data illustration.

• 出版日期2019-5
• 单位南开大学