Time Series Analysis – Solutions
Exercise 1
Consider the process defined for all \(t \in \mathbb{Z}\) by:
\[ X_t = \alpha X_{t-1} + \alpha^2 X_{t-2} + \varepsilon_t \]where \((\varepsilon_t)\) is a white noise with variance \(\sigma^2\) and \(\alpha \neq 0\).
1. Stationarity, causality, invertibility as a function of \(\alpha\)
The characteristic polynomial is:
\[ \phi(z) = 1 - \alpha z - \alpha^2 z^2 = 0 \quad \Longleftrightarrow \quad \alpha^2 z^2 + \alpha z - 1 = 0. \]Roots: \(z = \frac{-\alpha \pm \sqrt{\alpha^2 + 4\alpha^2}}{2\alpha^2} = \frac{-\alpha \pm |\alpha|\sqrt{5}}{2\alpha^2}\).
Two cases:
- If \(\alpha > 0\): roots are \(z_1 = \frac{-1+\sqrt{5}}{2\alpha}\), \(z_2 = \frac{-1-\sqrt{5}}{2\alpha}\) (negative). Stationarity requires \(|z_i| > 1\): for \(\alpha > 0\), \(|z_1| = \frac{\sqrt{5}-1}{2\alpha} > 1 \Rightarrow \alpha < \frac{\sqrt{5}-1}{2} \approx 0.618\). The negative root automatically satisfies \(|z_2|>1\) for small \(\alpha\). Thus for \(0 < \alpha < \frac{\sqrt{5}-1}{2}\) the process is stationary and causal.
- If \(\alpha < 0\): set \(\beta = -\alpha > 0\), the equation becomes \(1 + \beta z - \beta^2 z^2 =0\). Similar analysis shows stationarity for \(|\alpha| < \frac{\sqrt{5}-1}{2}\).
Therefore: Stationarity and causality \(\Longleftrightarrow\) \(|\alpha| < \frac{\sqrt{5}-1}{2} \approx 0.618\).
Invertibility: the MA part is just \(\varepsilon_t\) (pure AR), so it is always invertible (no MA polynomial).
2. Prove \(\operatorname{cov}(\varepsilon_n, X_{n-h}) = \sigma^2\) for \(h \ge 0\) under causality
Using the MA(\(\infty\)) representation (causality): \(X_t = \sum_{i=0}^\infty \psi_i \varepsilon_{t-i}\) with \(\psi_0 = 1\). Then
\[ \operatorname{cov}(\varepsilon_n, X_{n-h}) = \operatorname{cov}\left(\varepsilon_n, \sum_{i=0}^\infty \psi_i \varepsilon_{n-h-i}\right) = \sum_{i=0}^\infty \psi_i \underbrace{\operatorname{cov}(\varepsilon_n, \varepsilon_{n-h-i})}_{= \sigma^2 \delta_{0, h+i}}. \]The only non‑zero term occurs when \(h+i=0 \Rightarrow i = -h\). Since \(i\ge0\), this requires \(h=0\) and \(i=0\): then \(\operatorname{cov}(\varepsilon_n, X_n) = \psi_0 \sigma^2 = \sigma^2\). For \(h>0\) the covariance is zero.
(Note: the statement in the problem likely intends \(h=0\); otherwise it is 0.)
3. Compute \(E(X_t^2)\) and \(E(X_t X_{t-h})\)
Let \(\gamma(h) = E(X_t X_{t-h})\) (mean zero if stationary; if not, center first). By the AR(2) equation:
\[ \gamma(0) = \alpha^2 \gamma(0) + \alpha^4 \gamma(0) + 2\alpha^3 \gamma(1) + \sigma^2, \] \[ \gamma(1) = \alpha \gamma(0) + \alpha^2 \gamma(1) \quad \Rightarrow \quad \gamma(1) = \frac{\alpha}{1-\alpha^2} \gamma(0), \] \[ \gamma(2) = \alpha \gamma(1) + \alpha^2 \gamma(0), \] and for \(h\ge 2\): \(\gamma(h) = \alpha \gamma(h-1) + \alpha^2 \gamma(h-2)\).4. Forecast \(X_{T+1}\) and \(X_{T+2}\)
Optimal linear forecast (conditional expectation):
\[ \hat{X}_{T+1} = \alpha X_T + \alpha^2 X_{T-1}, \] \[ \hat{X}_{T+2} = \alpha \hat{X}_{T+1} + \alpha^2 X_T = \alpha(\alpha X_T + \alpha^2 X_{T-1}) + \alpha^2 X_T = (\alpha^2 + \alpha^2)X_T + \alpha^3 X_{T-1} = 2\alpha^2 X_T + \alpha^3 X_{T-1}. \]5. Variance of forecast errors
One-step ahead error: \(e_{T+1} = X_{T+1} - \hat{X}_{T+1} = \varepsilon_{T+1}\), so \(\operatorname{Var}(e_{T+1}) = \sigma^2\).
Two-step ahead: \(e_{T+2} = X_{T+2} - \hat{X}_{T+2} = \varepsilon_{T+2} + \alpha \varepsilon_{T+1}\), variance = \(\sigma^2(1 + \alpha^2)\).
6. Numerical values for \(\alpha = 0.5\), \(\sigma^2 = 1\)
Using \(\gamma(1) = \frac{0.5}{1-0.25}\gamma(0) = \frac{0.5}{0.75}\gamma(0) = \frac{2}{3}\gamma(0)\).
From \(\gamma(0) = \alpha^2\gamma(0)+\alpha^4\gamma(0)+2\alpha^3\gamma(1)+\sigma^2\):
\[ \gamma(0) = 0.25\gamma(0) + 0.0625\gamma(0) + 2\cdot 0.125 \cdot \frac{2}{3}\gamma(0) + 1, \] \[ \gamma(0)\left(1 - 0.3125 - \frac{0.5}{3}\right) = 1, \] \[ \gamma(0)\left(0.6875 - 0.166666...\right) = 1 \quad \Rightarrow \quad \gamma(0)\times 0.5208333 = 1 \quad \Rightarrow \quad \gamma(0) \approx 1.92. \] Then \(\gamma(1) = \frac{2}{3}\times 1.92 = 1.28\), and \(\gamma(2) = 0.5\gamma(1) + 0.25\gamma(0) = 0.64 + 0.48 = 1.12\).7. Yule–Walker equations
For \(h\ge 1\): \(\gamma(h) = \alpha \gamma(h-1) + \alpha^2 \gamma(h-2)\). With \(\rho(h)=\gamma(h)/\gamma(0)\):
\[ \rho(1) = \frac{\alpha}{1-\alpha^2}, \quad \rho(2) = \alpha \rho(1) + \alpha^2, \quad \rho(h) = \alpha \rho(h-1) + \alpha^2 \rho(h-2),\ h\ge 2. \]8. Proof that \(r(h) = 0\) for \(h>p\) and \(r(p)\neq 0\)
For an AR(p) process, the partial autocorrelation \(r(h)\) is the last coefficient in the projection of \(X_t\) onto \(X_{t-1},\dots,X_{t-h}\). For \(h > p\), the true model uses only lags up to \(p\), so the coefficient of \(X_{t-h}\) is zero. For \(h=p\), the coefficient equals \(\phi_p \neq 0\). Hence:
\[ r(h) = \begin{cases} 0, & h > p \\ \neq 0, & h = p \end{cases} \]This is a defining property of AR(p) processes.
Exercise 2
Consider an ARMA(1,1) process, stationary and invertible:
\[ (1 - \tfrac{1}{2}L)(1 - \tfrac{1}{5}L) x_t = (1 - \tfrac{1}{3}L) \varepsilon_t \]where \(\varepsilon_t\) is white noise with variance \(\sigma_\varepsilon^2\).
1. AR(\(\infty\)) representation
Since invertible (\(|\theta_1|=1/3<1\)), we write \(\Pi(L)x_t = \varepsilon_t\) with \(\Pi(L)=\Phi(L)/\Theta(L)\):
\[ \Phi(L) = 1 - \frac{7}{10}L + \frac{1}{10}L^2,\qquad \Theta(L)=1-\frac{1}{3}L. \] \[ \Pi(L) = \left(1 - \frac{7}{10}L + \frac{1}{10}L^2\right) \sum_{j=0}^\infty \left(\frac{L}{3}\right)^j. \]Coefficients:
\[ \pi_0 = 1,\quad \pi_1 = -\frac{7}{10} + \frac{1}{3} = -\frac{21}{30} + \frac{10}{30} = -\frac{11}{30}, \] \[ \pi_j = \frac{1}{3^j} - \frac{7}{10}\cdot\frac{1}{3^{j-1}} + \frac{1}{10}\cdot\frac{1}{3^{j-2}} = -\frac{1}{5\cdot 3^j},\quad j\ge 2. \] \[ \boxed{x_t - \frac{11}{30}x_{t-1} - \sum_{k=2}^{\infty} \frac{1}{5\cdot 3^k} x_{t-k} = \varepsilon_t} \]2. Autocovariances
The autocovariance \(\gamma(k)\) satisfies the homogeneous recurrence for \(k\ge 2\):
\[ \gamma(k) - \frac{7}{10}\gamma(k-1) + \frac{1}{10}\gamma(k-2) = 0. \]Characteristic roots: \(r_1 = 1/2,\ r_2 = 1/5\). Hence \(\gamma(k) = A(1/2)^k + B(1/5)^k\).
Initial conditions from Yule–Walker:
\[ \gamma(0) - \frac{7}{10}\gamma(1) + \frac{1}{10}\gamma(2) = \sigma_\varepsilon^2(1-\frac{1}{3}\psi_1),\quad \psi_1 = \frac{11}{30}, \] \[ \gamma(1) - \frac{7}{10}\gamma(0) + \frac{1}{10}\gamma(1) = -\frac{1}{3}\sigma_\varepsilon^2. \]Solving gives explicit \(A,B\) in terms of \(\sigma_\varepsilon^2\).
3. One-step ahead forecast \(\hat{x}_{t+1}\)
From the ARMA(1,1) form: \((1 - \frac{7}{10}L + \frac{1}{10}L^2)x_t = (1-\frac{1}{3}L)\varepsilon_t\)
\[ x_{t+1} = \frac{7}{10}x_t - \frac{1}{10}x_{t-1} + \varepsilon_{t+1} - \frac{1}{3}\varepsilon_t. \]Taking conditional expectation at time \(t\): \(\hat{x}_{t+1} = \frac{7}{10}x_t - \frac{1}{10}x_{t-1} - \frac{1}{3}\varepsilon_t\).
The innovation \(\varepsilon_t\) can be expressed via the AR(\(\infty\)) representation:
\[ \varepsilon_t = x_t - \frac{11}{30}x_{t-1} - \sum_{k=2}^{\infty} \frac{1}{5\cdot 3^k} x_{t-k}. \]Thus \(\hat{x}_{t+1}\) is a linear combination of present and past \(x\)’s.
4. Forecast function for any horizon \(h\ge 1\)
For \(h\ge 2\): projecting the ARMA equation onto \(\mathcal{F}_t\) yields
\[ \hat{x}_{t+h} = \frac{7}{10}\hat{x}_{t+h-1} - \frac{1}{10}\hat{x}_{t+h-2}, \quad h\ge 2, \]with \(\hat{x}_{t}=x_t\) and \(\hat{x}_{t+1}\) given above. The homogeneous solution:
\[ \hat{x}_{t+h} = C_1\left(\frac{1}{2}\right)^h + C_2\left(\frac{1}{5}\right)^h, \]where \(C_1, C_2\) are determined by \(x_t\) and \(\hat{x}_{t+1}\). For example,
\[ C_1 = \frac{5}{3}x_t - \frac{5}{3}\hat{x}_{t+1},\quad C_2 = -\frac{2}{3}x_t + \frac{5}{3}\hat{x}_{t+1}. \]Hence the forecast decays as a mixture of two geometric rates \(1/2\) and \(1/5\).
(Illustrative: the PACF cuts off after lag p)