Regression analysis

Lore & Background

The term 'regression' was coined by Francis Galton in the 19th century to describe the biological phenomenon of regression toward the mean, where heights of descendants of tall ancestors tend to regress down towards a normal average. Galton's work was later extended by Udny Yule and Karl Pearson to a more general statistical context, assuming a Gaussian joint distribution of response and explanatory variables. In the 1950s and 1960s, economists used electromechanical desk calculators to calculate regressions, with results sometimes taking up to 24 hours. Modern regression analysis is typically done with statistical and spreadsheet software on computers and handheld calculators.

Reader's Guide

Regression analysis is a foundational tool in statistics, used to model and analyze relationships between variables. Its most common form, linear regression, finds the line or hyperplane that best fits data according to criteria like ordinary least squares, which minimizes the sum of squared differences. The method is widely applied in prediction and forecasting, overlapping with machine learning, and can be used to infer causal relationships when carefully justified, especially with observational data. Historically, regression evolved from Newton's early averaging methods through the least squares developments by Legendre and Gauss, to Galton's biological concept of regression toward the mean. The work of Yule, Pearson, and Fisher expanded its statistical foundations, relaxing assumptions about data distributions. Regression methods continue to be an active research area, with modern developments including robust regression, nonparametric regression, Bayesian methods, and causal inference. Its significance lies in providing a rigorous framework for understanding how independent variables influence a dependent variable, enabling predictions and causal insights across fields from astronomy to economics.

Did You Know?

• The term 'regression' was coined by Francis Galton to describe the biological phenomenon of regression toward the mean.
• In the 1950s and 1960s, economists used electromechanical desk calculators to calculate regressions, sometimes taking up to 24 hours for one result.
• R.A. Fisher assumed the conditional distribution of the response variable is Gaussian, but the joint distribution need not be.