Minimum Hellinger distance estimation of a regression function in a parametric family with a random design model



Journal Title

Journal ISSN

Volume Title



There is a vast number of methods to estimate parameters of a statistical model based on maximizers and minimizers. These methods compete each other based on their properties such as unbiasness, robustness and efficiency. Some parameter estimation methods work best for specific models and fail when the underlying distribution undergoes even a slightest change due to the lack of robustness. Beran proposed an estimator based on Minimum Hellinger Distance (MHD) method that turned out to be both efficient and robust. Here we exploit his idea in the context of regression estimation. We consider a regression problem with random design where the regression function is defined on an arbitrary measurable space and is assumed to belong to a parametric family where parameter is a compact subset of the real line. In this random design model, the design variable is drawn form an unknown, completely arbitrary probability distribution on the design space. The error variable is assumed to have a known density with a finite second moment and zero mean. Moreover, we assume that the design variable and the error variable are stochastically independent. Summarizing, the response variable of the regression model turns out to have a density which is a convolution of the error distribution and the distribution of the design variable where the parameter is a compact set. In the estimation procedure, two different estimators for the density of the response variable will play a role. One is entirely nonparametric and automatically adjusts to the specific parameter at hand. The other one, however, is tailored to a specific parametric value. The MHD estimator for the parameter is now obtained as the minimizer of Hellinger distance between these two. After some elementary properties of the proposed MHD estimator, we prove consistency and asymptotic normality.



Hellinger distance, Regression function, Parametric space, Random design