Xycoon logo
Least Squares
Home    Site Map    Site Search    Free Online Software    
horizontal divider
vertical whitespace

Statistical Inference with Ordinary Least Squares

[Home] [Up] [Introduction] [Criterion] [OLS] [Assumptions] [Multiple Regression] [OLS] [MLE] [Inference]

II.I.4 Statistical inference with Ordinary Least Squares (OLS)

a. Mathematical expectation and variance of parameters

In order to be able to say something about the population parameters (of the real mathematical model), based only on the sample observations, it is imperative to compute the expectation and the variance of both estimated parameters.

The expectation of the estimated constant term can be derived as follows

Statistical Inference with Ordinary Least Squares



It is quite easy to derive the variance of the constant term as


Now consider the derivation of the expectation of the estimated parameter




The derivation of the variance is quite similar to (II.I.4-4)




From this analysis we conclude that in order to reduce the variance of the estimated parameters we should ensure that:


(a) the number of sample data should be large because of eq. (II.I.4-3);


(b) the (constant) variance of the endogenous variable should be relatively small (see eq. (II.I.4-3) and eq. (II.I.4-9));


(c) the range of the exogenous variable should be large because of eq. (II.I.4-9).

Remark that (a) and (c) is not only true in simple regression but also in all other econometric regressions (time series and cross-sectional data), multivariate statistic techniques, statistic time series analyses, random experiments, and even in controlled experiments (this only applies to (c) ).

Furthermore, it can be concluded from eq. (II.I.4-2)and eq. (II.I.4-6)that OLS for simple regression yields unbiased estimates for both parameters.

b. Confidence intervals for the parameters

In order to find the t statistic we first derive the Z transformation of the estimated value of


where the unobservable s is replaced by the sample variance since


so that by definition


Furthermore, the 95% confidence interval for any parameter is given by




represents the limit value of according to the students t-distribution (for the 5% significance level).

The confidence interval for a can be found in just the same way (cfr. (II.I.4-10) to (II.I.4-14)).

c. Forecasting errors

If the mean forecast is considered, a suitable confidence interval should be derived.

First we note


(we say: the mean estimator (II.I.4-15) is unbiased).

Additionally, the expression for the variance of the mean estimator is found as


Example of interpolation confidence interval

It is obvious from (II.I.4-16) that the forecast performance depends on: the variance of the endogenous variable, the sample size, the range of the exogenous variable, and x0; the distance between the forecast origin and the mean of the exogenous variable.

If however, an individual estimation of Y at origin t = o (o = origin) has to be performed, the variance should be added to (II.I.4-16)


vertical whitespace

Multiple Regression
horizontal divider
horizontal divider

© 2000-2022 All rights reserved. All Photographs (jpg files) are the property of Corel Corporation, Microsoft and their licensors. We acquired a non-transferable license to use these pictures in this website.
The free use of the scientific content in this website is granted for non commercial use only. In any case, the source (url) should always be clearly displayed. Under no circumstances are you allowed to reproduce, copy or redistribute the design, layout, or any content of this website (for commercial use) including any materials contained herein without the express written permission.

Information provided on this web site is provided "AS IS" without warranty of any kind, either express or implied, including, without limitation, warranties of merchantability, fitness for a particular purpose, and noninfringement. We use reasonable efforts to include accurate and timely information and periodically updates the information without notice. However, we make no warranties or representations as to the accuracy or completeness of such information, and it assumes no liability or responsibility for errors or omissions in the content of this web site. Your use of this web site is AT YOUR OWN RISK. Under no circumstances and under no legal theory shall we be liable to you or any other person for any direct, indirect, special, incidental, exemplary, or consequential damages arising from your access to, or use of, this web site.

Contributions and Scientific Research: Prof. Dr. E. Borghers, Prof. Dr. P. Wessa
Please, cite this website when used in publications: Xycoon (or Authors), Statistics - Econometrics - Forecasting (Title), Office for Research Development and Education (Publisher), http://www.xycoon.com/ (URL), (access or printout date).

Comments, Feedback, Bugs, Errors | Privacy Policy