2. Mathematical Formalism & Model Notation

To work effectively with ARIMA and SARIMA models, we utilize the Backshift Operator (also called the Lag Operator), denoted as $B$.

The Backshift Operator shifts a time series observation back by one time period:

$$B y_t = y_{t-1}$$
Applying the operator $k$ times shifts the series back by $k$ periods:
$$B^k y_t = y_{t-k}$$

2.1 The ARIMA Model: $\text{ARIMA}(p, d, q)$

An ARIMA model is parameterized by three non-negative integers:

• $p$: Order of the AutoRegressive (AR) component.
• $d$: Degree of non-seasonal differencing required for stationarity.
• $q$: Order of the Moving Average (MA) component.

Structural Equations

1. AutoRegressive Polynomial: $$\phi(B) = 1 - \phi_1 B - \phi_2 B^2 - \dots - \phi_p B^p$$

2. Moving Average Polynomial:

$$\theta(B) = 1 + \theta_1 B + \theta_2 B^2 + \dots + \theta_q B^q$$

3. Differencing Operator:
$$\nabla^d = (1 - B)^d$$

General Mathematical Form

Combining these terms yields the compact notation for an $\text{ARIMA}(p, d, q)$ model:

$$\phi(B)(1 - B)^d y_t = \theta(B) \varepsilon_t$$
Where:

• $y_t$ is the observed value at time $t$.
• $\varepsilon_t$ is a white noise error term, assuming $\varepsilon_t \sim \mathcal{N}(0, \sigma^2)$.

2.2 The SARIMA Model: $\text{SARIMA}(p, d, q)(P, D, Q)_s$

A SARIMA model adds four seasonal parameters to the standard ARIMA notation:

• $P$: Order of the Seasonal AutoRegressive (SAR) component.
• $D$: Degree of Seasonal Differencing.
• $Q$: Order of the Seasonal Moving Average (SMA) component.
• $s$: Length of the seasonal cycle/periodicity (e.g., $s=12$ for monthly data with annual seasonality).

Structural Equations

1. Seasonal AutoRegressive Polynomial:
$$\Phi(B^s) = 1 - \Phi_1 B^s - \Phi_2 B^{2s} - \dots - \Phi_P B^{Ps}$$

2. Seasonal Moving Average Polynomial:
$$\Theta(B^s) = 1 + \Theta_1 B^s + \Theta_2 B^{2s} + \dots + \Theta_Q B^{Qs}$$

3. Seasonal Differencing Operator:
$$\nabla_s^D = (1 - B^s)^D$$

General Mathematical Form

Multiplying the non-seasonal and seasonal operators together yields the full expanded algebraic expression for $\text{SARIMA}(p, d, q)(P, D, Q)_s$:
$$\phi(B) \Phi(B^s) (1 - B)^d (1 - B^s)^D y_t = \theta(B) \Theta(B^s) \varepsilon_t$$

3. Structural Comparison Matrix

4. Selecting the Seasonal Period ($s$)

The seasonal period $s$ represents the number of time steps contained within a single recurring cycle. Proper identification depends on both the underlying data collection frequency and the phenomenon being measured:

• Hourly Data ($s = 24$): Captures daily cycles (e.g., hourly power grid load).
• Daily Data ($s = 7$): Captures day-of-week seasonality (e.g., daily restaurant traffic).
• Monthly Data ($s = 12$): Captures annual seasonality (e.g., ice cream sales, monthly rainfall).
• Quarterly Data ($s = 4$): Captures annual corporate reporting cycles (e.g., GDP figures).

5. Decision Framework: When to Use ARIMA vs. SARIMA

Choosing between ARIMA and SARIMA requires evaluating the underlying structure of the time series.

6. End-to-End Model Identification & Implementation

Stage 1: Plot the Time Series

Visually inspect the series to identify overall trends, structural breaks, heteroscedasticity (changing variance), and repeating seasonal cycles.

Stage 2: Check & Achieve Stationarity

A stationary time series has a constant mean, variance, and autocorrelation structure over time.

Statistical Verification: Perform the Augmented Dickey-Fuller (ADF) Test.

Stage 3: Order Identification using ACF and PACF

Analyze the Autocorrelation Function (ACF) and Partial Autocorrelation Function (PACF) plots on the stationary series to select model orders:

Non-Seasonal Component Rules:

• Identifying $q$ (MA order): Look at the ACF plot. If the ACF cuts off sharply after lag $q$, while the PACF dies out exponentially or as a damped sine wave, set the MA order to $q$.
• Identifying $p$ (AR order): Look at the PACF plot. If the PACF cuts off sharply after lag $p$, while the ACF dies out exponentially or as a damped sine wave, set the AR order to $p$.
• Mixed Models ($p, q > 0$): If both ACF and PACF decay gradually, test small combinations of $p$ and $q$ (typically $p, q \le 2$).

Seasonal Component Rules ($P, Q, D$):

• Identify $D$: Use $D=1$ if the series displays persistent seasonal patterns after regular differencing.
• Identify $P$ (Seasonal AR order): Look at PACF spikes specifically at seasonal lags ($s, 2s, 3s, \dots$). A sharp cutoff after lag $P \cdot s$ indicates a seasonal AR term of order $P$.
• Identify $Q$ (Seasonal MA order): Look at ACF spikes specifically at seasonal lags ($s, 2s, 3s, \dots$). A sharp cutoff after lag $Q \cdot s$ indicates a seasonal MA term of order $Q$.

Stage 4: Fit the Model

Estimate model coefficients using Maximum Likelihood Estimation (MLE) or Non-Linear Least Squares. Compare competing model specifications using information criteria such as the Akaike Information Criterion (AIC) or Bayesian Information Criterion (BIC):
$$\text{AIC} = 2k - 2\ln(\hat{L})$$
Where $k$ is the number of estimated parameters and $\hat{L}$ is the maximized value of the likelihood function. Lower AIC/BIC values indicate a better balance between fit and model simplicity.

Stage 5: Model Diagnostics (Validation)

Check whether the model assumptions hold true by analyzing the residual error terms $e_t = y_t - \hat{y}_t$:

• Residual ACF/PACF: Residuals must show no significant autocorrelation at any lag.
• Ljung-Box Test:
• $H_0$: Residuals are independently distributed (white noise).
• Goal: A $p\text{-value} > 0.05$ confirms that residuals resemble white noise.
• Normality Check: Analyze a Normal Q-Q plot or perform a Jarque-Bera test to verify that residuals follow a zero-mean normal distribution $\mathcal{N}(0, \sigma^2)$.

Stage 6: Forecast & Validate

Generate out-of-sample forecasts and compute performance metrics such as Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), or Mean Absolute Percentage Error (MAPE) against a hold-out test set.

7. Summary Checklist & Pro-Tips

Pro-Tip: Always start with a simple model (e.g., $\text{ARIMA}(1,d,1)$ or $\text{SARIMA}(1,d,1)(1,D,1)_s$), evaluate its residual diagnostics, and only add higher-order terms ($p, q, P, Q$) if residual autocorrelation persists. Over-parameterization leads to overfitting and poor out-of-sample generalization.