Remember the curse of dimensionality: the case of goodness-of-fit testing in arbitrary dimension
Files
First author draft
Date
2018
Authors
Arias-Castro, Ery
Pelletier, Bruno
Saligrama, Venkatesh
Version
OA Version
Citation
Ery Arias-Castro, Bruno Pelletier & Venkatesh Saligrama (2018) Remember the curse of dimensionality: the case of goodness-of-fit testing in arbitrary dimension, Journal of Nonparametric Statistics, 30:2, pp. 448-471, https://doi.org/10.1080/10485252.2018.1435875
Abstract
Despite a substantial literature on nonparametric two-sample goodness-of-fit testing in arbitrary dimensions, there is no mention there of any curse of dimensionality. In fact, in some publications, a parametric rate is derived. As we discuss below, this is because a directional alternative is considered. Indeed, even in dimension one, Ingster, Y. I. [(1987). Minimax testing of nonparametric hypotheses on a distribution density in the l_p metrics. Theory of Probability & Its Applications, 31(2), 333–337] has shown that the minimax rate is not parametric. In this paper, we extend his results to arbitrary dimension and confirm that the minimax rate is not only nonparametric, exhibits but also a prototypical curse of dimensionality. We further extend Ingster's work to show that the chi-squared test achieves the minimax rate. Moreover, we show that the test adapts to the intrinsic dimensionality of the data. Finally, in the spirit of Ingster, Y. I. [(2000). Adaptive chi-square tests. Journal of Mathematical Sciences, 99(2), 1110–1119], we consider a multiscale version of the chi-square test, showing that one can adapt to unknown smoothness without much loss in power.