Auto Correlation Function and Partial Auto Correlation Function

The autocovariance matrix and autocorrelation matrix associated with a stochastic stationary process

is always positive definite, which can be easily shown since a linear combination of the stochastic variable

has a variance of

which is always positive.

Suppose that T=3, then we find

or

Bartlett proved that the variance of autocorrelation of a stationary normal stochastic process can be formulated as

This expression can be shown to be approximated by

if the autocorrelation coefficients decrease exponentially like

If it can be assumed that the autocorrelations for i > q (a natural number) are equal to zero, Bartlett’s variance formula can be shown to be approximated by

which is the so called large-lag variance. Now it is possible to vary q from 1 to any desired integer number of autocorrelations, replace the theoretical correlations by their sample estimates, and compute the square root to find the standard deviation of the sample autocorrelations.

Note that the standard deviation of a autocorrelation coefficient is almost always approximated by

The covariances between autocorrelation coefficients have also been deduced by Bartlett

which is a good indicator for dependencies between autocorrelations. Inter-correlated autocorrelations can seriously distort the pattern of the autocorrelation function.

It is possible to remove these distorting inter-correlations. The ‘corrected’ Auto Correlation Function is called “Partial Auto Correlation Function” (PACF).

The partial autocorrelation coefficients are defined as the last coefficient of a partial autoregression equation of order k

It is obvious that there exists a relationship between the PACF and the ACF because the above relationship can be rewritten

or (on taking expectations and dividing by the variance)

Sometimes this is written in matrix formulation according to the so-called Yule-Walker relations

or simply

which can be solved with Cramer's Rule

A practical numerical estimation algorithm for the PACF is given by Durbin

with

The standard error of a partial autocorrelation coefficient for k > p (where p is the order of the autoregressive data generating process - explained later) is given by