Regression splines with free knots vs. penalized splines : A comparative study

Abstract

Most non-parametric approaches to the curve fitting problem are based on either penalized spline regression with fixed knots, or smoothing splines with knots located at each design point. As a result, they have a tendency to produce models with large numbers of parameters, even though the underlying model is fairly simple. This research proposes a new algorithm that fits a B-spline function to a set of data points with independent and identically distributed random noise. The algorithm automatically chooses (i) the order of the regression spline, (ii) the number of interior knots, and (iii) the locations of the knots. The procedure is based on minimizing the residual sum of squares for the curve being fitted. The performance of various model selection information criteria is discussed, and the method is compared with penalized spline regression. Simulation results show that the proposed method is superior in terms of the bias and the mean summed squared error of the fit. Overall results highlight the importance of selecting proper knot locations when fitting a spline function. A novel extension of the methodology considers estimation of a B-spline function by re-casting it in the framework of a nonlinear mixed effects model. An extension of the proposed algorithm to models with auto-correlated errors is briefly discussed.