Full Note on Augmented Dickey-Fuller Test

In time-series forecasting, statistical modeling, and econometrics, understanding the underlying behavior of your data is paramount. Before deploying complex predictive models like ARIMA or LSTM neural networks, data scientists must answer a fundamental question: Is the data stationary? The Dickey-Fuller (DF) test is the statistical backbone used to answer this question. This guide explores the core philosophy of stationarity, the mathematics behind the unit root problem, and provides five rigorous, step-by-step mathematical examples based on real-world scenarios.

Part I: The Core Concepts

1. The Philosophy of Stationarity

• Constant Mean: The series fluctuates around a consistent, unchanging long-term average.
• Constant Variance (Homoscedasticity): The amplitude or spread of the fluctuations does not grow or shrink systematically over time.
• Time-Independent Autocovariance: How the series correlates with its own past depends only on the lag (the time gap between data points), not the actual point in time.

🧠 The "Dog on a Leash" Analogy
To understand stationarity, imagine a man walking his dog on a leash through a park. The man walks in a perfectly straight line (the mean).

• Stationary: The dog might pull to the left to sniff a tree, or to the right to chase a squirrel, but because of the leash, the dog will always be pulled back to the man's side. The dog's path fluctuates, but it always reverts to a constant center.
• Non-Stationary: Now imagine a stray dog in the same park. It follows a scent to the left, then another scent further left, then sleeps, then wanders right. It has no leash. It wanders aimlessly, and its future location cannot be reliably predicted based on a central "mean."

🌍 Real-World Examples
Stationary Examples (Mean-Reverting):

• A healthy person's resting heart rate: It might jump if they hear a loud noise, but it will quickly revert to around 70 BPM.
• Daily temperature in Dhaka during January: It fluctuates day by day, but always hovers around 18°C to 25°C. It doesn't randomly wander to 50°C.
• The noise on a blank TV channel (Static): The static is entirely random, but consistently centered at zero intensity.

Non-Stationary Examples (Wandering/Trending):

• Bitcoin Prices: There is no "true" baseline price that Bitcoin always returns to. It wanders based on market shocks.
• A country's GDP over 50 years: It trends upward over time. The mean in 1990 is vastly different from the mean in 2020.
• Your cumulative wealth: As you earn and spend, your total wealth wanders and hopefully trends upward. It does not revert to zero every month.

Why does this matter? If these conditions fail (meaning the data is non-stationary), traditional predictive models will misinterpret random, wandering fluctuations as meaningful trends. This leads to spurious regressions—where models show strong statistical significance and high $R^2$ values for variables that have absolutely no real-world relationship.

2. The Unit Root Problem

To test for stationarity, we mathematically check for the presence of a Unit Root.
Consider a basic First-Order Autoregressive model, or AR(1), which predicts today's value based on yesterday's value:

🧠 What does "Unit Root" actually mean? Think of it as "100% Memory"

The term "unit root" comes from complex polynomial math, but intuitively, it refers to the value of $\rho$ being exactly 1 (one unit). Let's look at the two scenarios:

Scenario A: The Leaky Bucket (Stationary, $\rho < 1$)
Imagine a bucket with a small hole in it. Yesterday, you poured some water into it ($y_{t-1}$). Because of the hole, only 70% of yesterday's water remains today ($\rho = 0.7$). Today, it rains a random amount ($\epsilon_t$).
The equation is $y_t = 0.7 y_{t-1} + \epsilon_t$. Because it constantly leaks, the bucket will never infinitely overflow. It will eventually stabilize at a constant average level based on the average rainfall. This is stationary. There is no unit root.

Scenario B: The Perfect Bucket (Non-Stationary, $\rho = 1$)
Now imagine a perfect bucket with no leaks. 100% of yesterday's water is preserved today ($\rho = 1$). Today, it rains a random amount.
The equation becomes $y_t = 1.0 y_{t-1} + \epsilon_t$. Because it has perfect "memory" of yesterday's water and never leaks, random daily rain will eventually cause the bucket to wander to completely empty or infinitely overflow. This is a Random Walk. It possesses a Unit Root and is Non-Stationary.
If $\rho = 1$, the mathematical behavior is a random walk. It has an infinite variance over time; it can wander anywhere and never reliably reverts to a mean.

3. The Mechanics of the Dickey-Fuller Test

$$y_t - y_{t-1} = \rho y_{t-1} - y_{t-1} + \epsilon_t$$
$$\Delta y_t = (\rho - 1) y_{t-1} + \epsilon_t$$

Let's create a new variable: $\delta = \rho - 1$. The testable equation becomes beautifully simple:
$$\Delta y_t = \delta y_{t-1} + \epsilon_t$$

Now, we set up our hypothesis test:

Null Hypothesis ($H_0$): $\delta = 0$. (If $\delta = 0$, then $\rho = 1$. The series has a unit root, meaning perfect memory, and is non-stationary).
Alternative Hypothesis ($H_a$): $\delta < 0$. (If $\delta < 0$, then $\rho < 1$. The series leaks past information, has no unit root, and is stationary).

Part II: Real-World Mathematical Examples

The Universal Formulas:
To solve these, we calculate the Ordinary Least Squares (OLS) estimate for $\delta$ and find its t-statistic.
Coefficient ($\delta$): $\delta = \frac{\sum (y_{t-1} \Delta y_t)}{\sum (y_{t-1})^2}$

Sum of Squared Errors (SSE): $SSE = \sum (\Delta y_t - \delta y_{t-1})^2$

Standard Error ($SE$): $SE(\delta) = \sqrt{\frac{SSE / (n-1)}{\sum (y_{t-1})^2}}$ (where $n-1 = 3$ for $4$ data pairs)

Test Statistic ($t$): $t = \frac{\delta}{SE(\delta)}$

Critical Value: We will use -1.95 (the rough critical value at a 5% significance level for a no-constant DF test).

Example 1: Land Surface Temperature (LST) Anomalies

You are using remote sensing data to track extreme daily LST anomalies (in °C) from a baseline mean. LST anomalies should theoretically revert to the mean (0) quickly if weather stabilizes.

Data ($y_t$): 2.0, -1.0, 1.5, -0.5, 1.0

Step by Step Solution:

1. Calculate Coefficient:

$\delta = \frac{\sum (y_{t-1} \Delta y_t)}{\sum (y_{t-1})^2}$ = $\frac{-12.25}{7.5} \approx -1.633$

2. Calculate Sum of Squared Errors (SSE):

$SSE = \sum (\Delta y_t - \delta y_{t-1})^2$ = $1.493$

3. Calculate Standard Error $SE$:

$SE(\delta) = \sqrt{\frac{SSE / (n-1)}{\sum (y_{t-1})^2}}$ (where $n-1 = 3$ for $4$ data pairs)

$SE(\delta) = \sqrt{\frac{1.493 / (4-1)}{7.5}} \approx 0.258$

4. Calculate Test Statistic ($t$):

$t = \frac{\delta}{SE(\delta)}$

$= \frac{-1.633}{0.258} \approx -6.33$

Conclusion: Because -6.33<-1.95, we reject $H_0$ or null hypothesis. The LST anomaly series is stationary, and has no unit root.

Example 2: Mobile Gaming Server Latency (Ping)

You are benchmarking server performance, tracking daily peak latency (ping in milliseconds) normalized around a baseline. Over a week, server load steadily increases without clearing.

Data ($y_t$): 2.0, 4.0, 3.0, 5.0, 6.0