Online Econometrics Textbook - Regression Extensions - Restricted Least Squares and Restricted Maximum Likelihood

Sometimes it is necessary to impose linear constraints on OLS estimation. This case of OLS estimation is called restricted least squares (RLS). Assume that the constraints are contained in

(III.V-1)

where b is the RLS parameter vector (K*1), r is a (H*1) vector, R is a (H*K) matrix, K is the number of parameters, and H is the number of restrictions.

A solution to RLS is only possible if H is smaller than K and if the rank of R is equal to H. Otherwise, two or more linear restrictions are linearly dependent or inconsistent.

Suppose K = 3, H = 2 and we wish to restrict the least squares estimates according to

(III.V-2)

then eq. (III.V-1) applied to (III.V-2) would look like

(III.V-3)

from which it can be clearly seen that R has rank H = 2.

The Lagrangian function can be deduced as follows

(III.V-4)

which can be derived w.r.t. the parameter vector and the Lagrange multiplier vector

(III.V-5)

(III.V-6)

On combining (III.V-5) and (III.V-6) we obtain

(III.V-7)

(III.V-8)

which is the RLS estimator.

The SSR are computed by the following formula

(III.V-9)

which can be written more explicitly

(III.V-10)

or on using X'(y - Xb) = 0

(III.V-11)

or simply

(III.V-12)

This result is very important since it can be used in tests for linear restrictions.

First we deduce from the RLS estimator (III.V-8) that

(III.V-13)

If we define

(III.V-14)

(for which proof is quite easy), we may write

(III.V-15)

Now since

(III.V-16)

(III.V-17)

Since

(III.V-18)

a general test for linear restrictions is thus given by

(III.V-19)

(it can be shown that the nominator of (III.V-19) is independent of the denominator).

Obviously, restrictions on parameters cannot only be imposed on least squares estimators. There is an equivalent approach with respect to MLE, the so-called restricted maximum likelihood estimation (RMLE). The greatest advantage of RMLE over RLS is that it is possible to impose nonlinear restrictions on the parameters. The most important disadvantage however might be the computational (numerical) complexity of RMLE.