# Lecture 08 (Feb 09, 2022)

The $ARMA(1,1)$ model properties: $\mathbb{E}[x_t] - \phi_1\mathbb{E}[x_{t-1}] = \phi_0$. This shows

$\mu = \mathbb{E}(x_t) = \frac{\phi_0}{1-\phi_1}$

which is the same as the AR(1) model.

For second order, assume $\phi_0 = 0$, we find

\begin{aligned} &\mathbb{E}(\epsilon_t(x_t-\phi_1x_{t-1})) = \mathbb{E}(\epsilon_t (\epsilon_t-\theta_1\epsilon_{t-1})) = \mathbb{E}(\epsilon_t^2) = \sigma^2 \\ &\Rightarrow \mathbb{E}\epsilon_t x_t = \sigma^2 \end{aligned}

The varaince is

\begin{aligned} \text{Var}(x_t) &= \text{Var}(\epsilon_t -\theta_1\epsilon_{t-1} + \phi_1x_{t-1})\\ &= \text{Var}(\epsilon_t) + \text{Var}(\phi_1x_{t-1}-\theta_1\epsilon_{t-1}) \\ &=\sigma^2 + \phi_1^2\text{Var}(x_{t-1}) + \theta_1\sigma^2 - 2\phi_1\theta_1\text{Cov}(x_{t-1},\epsilon_{t-1}) \end{aligned}

since $\text{Cov}(x_{t-1},\epsilon_{t-1})$ is $1$, we have

$\text{Var}(x_t) = \frac{\sigma^2(1+\theta_1^2-2\phi_1\theta_1)}{1-\phi_1^2}$

TO get $\gamma_l$, $l\geq 1$, compute

$\mathbb{E}(x_{t-l} (x_t - \phi_1x_{t-1})) = E[x_{t-l}(\epsilon_t-\theta_1\epsilon_{t-1})]$

This becomes

\begin{aligned} \gamma_l - \phi_1\gamma_{l-1} = - \theta_1\mathbb{E}x_{t-l}\epsilon_{t-1} \end{aligned}

for $l=1$, the equation is

$\gamma_1 - \phi_1\gamma_0 = - \theta_1\sigma^2$

for $l\geq 2$, we have

$\gamma_l = \phi_1\gamma_{l-1}$

thus, ACF is

$\rho_0 = 1,\quad \rho_1 = \frac{\theta_1\gamma_0 - \theta_1\sigma^2}{\gamma_0}, \quad \rho_l = \phi_1\rho_{l-1}$

The ACF does not cut off at any finite order.

Rmk: PACF of ARMA(1,1) also does not cutoff at any finite leg.

General, $ARMA(p,q)$ model:

$x_t - \sum_{i=1}^p\phi_ix_{t-i} = \phi_0 +\sum_{j=1}^q \theta_j\epsilon_{t-j}$

Write as

$(1-\phi_1B-\phi_2B^2-\dots - \phi_pB^2) x_t = \phi_0 + (1-\theta_1B - \theta_2B^2 -\dots \theta_qB^q)\epsilon_t$

abbrieviately, we write

$\phi(B) x_t = \phi_0 + \theta(B)\epsilon_t$

The two polynomials, $\theta(B)$ and $\phi(B)$ need to not have common factors. otherwise, the order $(p,q)$ of the model can be reduced. The MA part of the ARMA model does not affect the staitonarity of the model. thus, the stationary condition of $ARMA(p,q)$ should be the same as $AR(p)$ model.

We can divide $\phi(B)$ on both sides to get an $MA$ representation of the $ARMA$ model.

$\frac{\theta(B)}{\phi(B)} = \psi(B) = 1 + \psi_1 B + \psi_2 B^2 +\dots$

we can also get the $AR$ representation by dividing $\theta(B)$ on both sides

$\frac{\phi(B)}{\theta(B)} = \pi(B) = 1- \pi_1B -\pi_2B - \dots$

For example for $ARMA(1,1)$, $\phi(B) = 1-\phi_1B$ and $\theta(B) = 1-\theta_1B$,

\begin{aligned} &\frac{\theta(B)}{\phi(B)} = \frac{1-\theta_1B}{1-\phi_1B} = \frac{1-\phi_1B + \phi_1B - \theta_1B}{1-\phi_1B} \\ &=1+ \frac{(\phi_1-\theta_1)B}{1-\phi_1B} \\ &=1 + (\phi_1-\theta_1)B\frac{1-\phi_1B+\phi_1B}{1-\phi_1B} \\ &=1 + (\phi_1-\theta_1)B + (\phi_1-\theta_1)\phi_1 B^2\frac{1}{1-\phi_1B} \\ &=1 + (\phi_1-\theta_1)B + (\phi_1-\theta_1)\phi_1B^2 + (\phi_1-\theta_1)\phi_1^2B^3 + \dots \end{aligned}

Thus, $\psi_1 = \phi_1-\theta_1$, $\psi_2 = (\phi_1-\theta_1)\phi_1$, $\psi_3 = (\phi_1-\theta_1)\phi_1^2, \dots$.

Similarly,

$\pi(B) = \frac{1-\phi_1B}{1-\theta_1B} = 1 - (\theta_1-\phi_1)B - \theta_1(\theta_1-\phi_1)B^2 - \dots$

We can write the $ARMA(p,q)$ model as $AR(\infty)$ and $MA(\infty)$.

\begin{aligned} &\frac{\phi(B)}{\theta(B)} x_t = \frac{\phi_0}{\theta(B)} + \epsilon_t\\ &\pi(B)x_t = \frac{\phi_0}{1-\theta_1-\theta_2-\dots } + \epsilon_t \end{aligned}

where $\phi_0/\theta(B) = x$ for some constant $x$ but $B x = x$ for the constant $x$, we thus $\phi_0 = (1-\theta_1-\theta_2-\dots )x$ and we get $x$ as the first term above. In the AR representation, need $\pi_i\rightarrow 0$ for $i\rightarrow\infty$. This means $ARMA(p,q)$ is invertible.

We can also write the $ARMA(p,q)$ to $MA(\infty)$.

\begin{aligned} x_t &= \frac{\phi_0}{1-\phi_1-\phi_2-\dots} + \frac{\theta(B)}{\phi(B)} \epsilon_t\\ &=\frac{\phi_0}{1-\phi_1-\phi_2-\dots} + \epsilon_t + \psi_1\epsilon_{t-1}+\psi_2\epsilon_{t-2} + \dots \end{aligned}

Updated: