Linear Regression

This chapter is talking about linear regression, a very simple approach for supervised learning. It assumes that the regression function $E(Y|X)$ is linear in inputs $X_1, \dots, X_p$.

They are simple and often provide an adequate and interpretable description of how the inputs affect the output. For prediction purposes they can sometimes outperform fancier nonlinear models, especially in situations with small numbers of training cases, low signal-to-noise ratio or sparse data.