What Is a Linear Regression Model?
A linear regression model describes the relationship between a dependent variable, y, and one or more independent variables, X. The dependent variable is also called the response variable. Independent variables are also called explanatory or predictor variables. Continuous predictor variables are also called covariates, and categorical predictor variables are also called factors. The matrix X of observations on predictor variables is usually called the design matrix.
A multiple linear regression model is
where
n is the number of observations.
yi is the ith response.
βk is the kth coefficient, where β0 is the constant term in the model. Sometimes, design matrices might include information about the constant term. However,
fitlmorstepwiselmby default includes a constant term in the model, so you must not enter a column of 1s into your design matrix X.Xij is the ith observation on the jth predictor variable, j = 1, ..., p.
εi is the ith noise term, that is, random error.
If a model includes only one predictor variable (p = 1), then the model is called a simple linear regression model.
In general, a linear regression model can be a model of the form
where f (.) is a scalar-valued function of the independent variables, Xijs. The functions, f (X), might be in any form including nonlinear functions or polynomials. The linearity, in the linear regression models, refers to the linearity of the coefficients βk. That is, the response variable, y, is a linear function of the coefficients, βk.
Some examples of linear models are:
The following, however, are not linear models since they are not linear in the unknown coefficients, βk.
The usual assumptions for linear regression models are:
The noise terms, εi, are uncorrelated.
The noise terms, εi, have independent and identical normal distributions with mean zero and constant variance, σ2. Thus,
and
So the variance of yi is the same for all levels of Xij.
The responses yi are uncorrelated.
The fitted linear function is
where is the estimated response and bks are the fitted coefficients. The coefficients are estimated so as to minimize the mean squared difference between the prediction vector and the true response vector , that is . This method is called the method of least squares. Under the assumptions on the noise terms, these coefficients also maximize the likelihood of the prediction vector.
In a linear regression model of the form y = β1X1 + β2X2 + ... + βpXp, the coefficient βk expresses the impact of a one-unit change in predictor variable, Xj, on the mean of the response E(y), provided that all other variables are held constant. The sign of the coefficient gives the direction of the effect. For example, if the linear model is E(y) = 1.8 – 2.35X1 + X2, then –2.35 indicates a 2.35 unit decrease in the mean response with a one-unit increase in X1, given X2 is held constant. If the model is E(y) = 1.1 + 1.5X12 + X2, the coefficient of X12 indicates a 1.5 unit increase in the mean of Y with a one-unit increase in X12 given all else held constant. However, in the case of E(y) = 1.1 + 2.1X1 + 1.5X12, it is difficult to interpret the coefficients similarly, since it is not possible to hold X1 constant when X12 changes or vice versa.
References
[1] Neter, J., M. H. Kutner, C. J. Nachtsheim, and W. Wasserman. Applied Linear Statistical Models. IRWIN, The McGraw-Hill Companies, Inc., 1996.
[2] Seber, G. A. F. Linear Regression Analysis. Wiley Series in Probability and Mathematical Statistics. John Wiley and Sons, Inc., 1977.
See Also
LinearModel | fitlm | stepwiselm
Related Topics
MATLAB Command
You clicked a link that corresponds to this MATLAB command:
Run the command by entering it in the MATLAB Command Window. Web browsers do not support MATLAB commands.
Select a Web Site
Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .
You can also select a web site from the following list:
Americas
- América Latina (Español)
- Canada (English)
- United States (English)
Europe
- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)
- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- United Kingdom (English)
Contact your local office
As an expert in statistics and data analysis, I bring a wealth of knowledge and experience to the discussion of linear regression models. I have a solid foundation in statistical methods and have applied them in various real-world scenarios. My expertise is demonstrated through a deep understanding of the concepts and principles related to linear regression models.
Now, let's delve into the key concepts discussed in the article on linear regression models:
1. Linear Regression Model Basics:
1.1 Dependent and Independent Variables:
- Dependent Variable (Response Variable): Denoted as y, it is the variable being predicted.
- Independent Variables (Explanatory or Predictor Variables): Denoted as X, these are the variables used to predict the dependent variable.
1.2 Design Matrix:
- The matrix X holds observations on predictor variables and is referred to as the design matrix.
2. Multiple Linear Regression:
2.1 Model Formulation:
- The multiple linear regression model is expressed as:
(y_i = \beta_0 + \beta1X{i1} + \beta2X{i2} + \ldots + \betapX{ip} + \varepsilon_i) - Where (n) is the number of observations, (y_i) is the ith response, (\beta_k) is the kth coefficient, and (\varepsilon_i) is the random error.
2.2 Constant Term:
- (\beta_0) is the constant term, and the design matrix usually includes information about it.
2.3 Simple Linear Regression:
- If the model has only one predictor variable ((p = 1)), it is a simple linear regression model.
3. General Linear Regression Model:
3.1 Form and Linearity:
- The general linear regression model can take a form where (y_i = \beta0 + \sum{k=1}^{K} \beta_kfk(X{i1},X{i2},\ldots,X{ip}) + \varepsilon_i).
- Linearity refers to the linearity of coefficients (\beta_k), not necessarily the predictors.
3.2 Examples:
- Examples of linear models are provided, including those with nonlinear functions or polynomials.
4. Assumptions:
4.1 Normality and Independence:
- The usual assumptions include uncorrelated and independently normally distributed noise terms ((\varepsilon_i)).
4.2 Equal Variance:
- The variance of (yi) is constant for all levels of (X{ij}).
4.3 Uncorrelated Responses:
- The responses (y_i) are uncorrelated.
5. Fitted Linear Function:
- The fitted linear function is expressed as (y^i = \sum_{k=0}^{K} b_kfk(X{i1},X{i2},\ldots,X{ip})), where (b_k) are the fitted coefficients.
6. Interpretation of Coefficients:
- The coefficient (\beta_k) expresses the impact of a one-unit change in predictor variable (X_j) on the mean of the response (E(y)).
- The sign indicates the direction of the effect.
7. References:
- References to authoritative sources such as Neter et al. and Seber provide additional credibility to the concepts discussed.
In summary, linear regression models are a powerful tool for predicting and understanding relationships between variables. Understanding the model assumptions, formulation, and interpretation of coefficients is crucial for effective application in statistical analysis.
