In traditional parametric regression models, the functional form of the model is specified before the model is fit to data, and the object is to estimate the parameters of the model. In nonparametric regression, in contrast, the object is to estimate the regression function directly without specifying its form explicitly. In this appendix to Fox and Weisberg (2011), we describe how to fit several kinds of nonparametric-regression models in R, including scatterplot smoothers, where there is a single predictor; models for multiple regression; additive regression models; and generalized nonparametric-regression models that are analogs to generalized linear models.
Regression Models The traditional nonlinear regression model (described in the Appendix on nonlinear regression) where is a vector of parameters to be estimated, and x is a vector of predictors; the errors are assumed to be normally and independently distributed with mean 0 and constant variance 2 . The function ø m(x,, relating the average value of the response to the predictors, is specified in advance, as it is in a linear regression model.
There are several approaches to estimating nonparametric regression models, of which we will describe two: local polynomial regression and smoothing splines. With respect to implementation of these methods in R, there is an embarrassment of riches: Local polynomial regression is performed by the standard R functions lowess (locally weighted scatterplot smoother, for the simple-regression case) and loess (local regression, more generally). Simple-regression smoothing-spline estimation is performed by the standard R function smooth.spline. Generalized nonparametric regression by local likelihood estimation (of which local regression is a special case for models with normal errors) is implemented in the locfit (local fitting) package (Loader, 1999), which also performs density estimation. 2 Generalized additive models may be fit with Hastie and Tibshirani’s (1990) gam function (in the gam package), which uses spline or local-regression smoothers. The gam function in Wood’s (2000, 2001, 2006) mgcv package, which is part of the standard R distribution, also fits this class of models using spline smoothers, and features automatic selection of smoothing parameters. (The name of the package comes from the method employed to pick the smoothing parameters: multiple generalized cross-validation.) There are several other R package for nonparametric regression, including Bowman and Azzalini’s (1997) sm (smoothing) package, which performs local-regression and local-likelihood estimation, and which also includes facilities for nonparametric density estimation; and Gu’s (2000) gss (general smoothing splines) package, which fits various smoothing-spline regression and generalized regression models. This is not an exhaustive list!
2.1 Local Polynomial Regression
2.1.1 Simple Regression
It is typical to adjust ℎ so that each local regression includes a fixed proportion of the data; then, is called the span of the local-regression smoother. The larger the span, the smoother the result; in contrast, the larger the order of the local regressions 푝, the more flexible the smooth, so the span and the order of the local regressions can be traded off against one-another.
The process of fitting a local regression is illustrated in Figure 1, using the Canadian occupationalprestige data introduced in Chapter 2 of the text. We examine the regression of prestige on income, focusing initially on the observation with the 80th largest income value, 푥(80), represented in Figure 1 by the vertical solid line.2
A window including the 50 nearest -neighbors of (80) (i.e., for span = 50/102 ≈ 1/2) is shown in Figure 1a.
The tricube weights for observations in this neighborhood appear in Figure 1b.
2.1.2 Multiple Regression
Specifying degree=1 fits locally linear regressions; the default is degree=2 (i.e., locally quadratic regressions). To see the full range of arguments for the loess function, consult ?loess. The summary output includes the standard deviation of the residuals under the model and an estimate of the equivalent number of parameters (or degrees of freedom) employed by the model — in this case, about eight parameters. In contrast, a standard linear regression model would have employed three parameters (the constant and two slopes).
As in nonparametric simple regression, there are no parameters estimates: To see the result of the regression, we have to examine the fitted regression surface graphically, as in Figure 2, produced by the following R commands.
We employ the expand.grid function to create a data frame containing combinations of values of the two predictors, income and education; for each predictor, we take 25 values, evenly spaced along the range of the variable. Then, corresponding fitted values on the regression surface are computed by predict. These predicted values are reshaped into a 25 by 25 matrix, which is passed to the persp function, along with the values of the predictors (inc and ed) used to generate the regression surface. The arguments theta and phi to persp control the orientation of the plot; expand controls the relative length of the 푧 axis; and shade controls the shading of the plotted surface. See ?persp for details