Least squares regression
Also known as: least squares regression line, ordinary least squares
Least squares regression is a method for fitting a straight line to paired data by minimizing the sum of the squared vertical distances between the observed points and the line. The resulting line is the standard tool for predicting one variable from another.
For any line drawn through a scatterplot, each data point has a residual — the vertical distance from the point to the line, calculated as observed y minus predicted y. Squaring the residuals removes negative signs and penalizes large misses more heavily than small ones. The least squares regression line is the unique line that makes the sum of those squared residuals as small as possible.
The fitted line is written ŷ = a + bx, where b is the slope and a is the y-intercept. The slope can be computed as b = r(sᵧ / sₓ), using the correlation coefficient and the standard deviations of the two variables, and the intercept follows from the fact that the line always passes through the point of averages (x̄, ȳ). The slope is interpreted as the predicted change in y for a one-unit increase in x, and r², the coefficient of determination, gives the proportion of variation in y explained by the linear relationship.
Checking the fit matters as much as computing it. A residual plot should show no pattern; a curved or fanning residual plot means a straight line is the wrong model. Points with extreme x values have high leverage and can pull the line noticeably, and predicting outside the range of the observed data — extrapolation — is unreliable no matter how good the fit looks.
AP Statistics tests least squares regression in both of its major units. The two-variable data unit covers computing and interpreting the line, its slope, r, r², and residual plots, while the inference unit builds on it with confidence intervals and significance tests for the slope of the regression line.
Key takeaways
- The least squares line minimizes the sum of squared residuals, where a residual is observed y minus predicted y.
- The line is written ŷ = a + bx, with slope b = r(sᵧ / sₓ), and always passes through (x̄, ȳ).
- r² gives the proportion of variation in y explained by the linear model.
- A patterned residual plot signals that a linear model is not appropriate.
- Extrapolating beyond the observed range of x produces unreliable predictions.