Professor James D. Hamilton requires no introduction, having
been one of the most important researchers in time series econometrics for
decades.

Over the past few years, Hamilton has been working on a
paper calling on applied economists to abandon the ubiquitous Hodrick-Prescott
Filter and replace it with a much simpler method of extracting trend and cycle
information from a time series.

This paper has become popular, and a number of our users
have asked how to replicate it in EViews. One of our users, Greg Thornton, has
written an EViews add-in (called Hamilton) that performs Hamilton’s method. However, given its relative simplicity, we
thought we’d use a blog post to show manual calculation of the method and
replicate the results in Hamilton’s paper.

## The Hodrick-Prescott Filter

The HP filter is a mainstay of modern applied macroeconomic
analysis. It is used extensively to isolate trend and cycle components from a
time series. By isolating and removing
the cyclical component, you are able to analyze the long-term effects of or on
a variable without worrying about the impact of short term fluctuations. In macroeconomics
this is especially useful since many macroeconomic variables suffer from
business-cycle fluctuations.

Mathematically, the HP filter is a two-sided linear filter
that computes the smoothed series $s$
of $y$ by minimizing the variance of $y$ around $s$, subject to a penalty that constrains the second difference of $s$. That is, the HP filter chooses $s$ to minimize:

$$\sum_{t=1}^T\left(y_t - s_t\right)^2 + \lambda \sum_{t=2}^{T-1}\left((s_{t+1} - s_t) - (s_t - s_{t-1})\right)^2$$

The arbitrary smoothing parameter $\lambda$ controls the
smoothness of the series $s$. The
larger the $\lambda$, the smoother the series. As $\lambda=\infty$, $s$ approaches a linear trend.

## Hamilton’s Criticisms of the HP Filter

Hamilton outlines three main criticisms of the HP filter:

*The HP filter produces series with spurious dynamic relations that have no basis in the underlying data-generating process.**Filtered values at the end of the sample are very different from those in the middle, and are also characterized by spurious dynamics.**A statistical formalization of the problem typically produces values for the smoothing parameter vastly at odds with common practice.*

## Hamilton’s Method

Hamilton proposes an alternative to the HP Filter that uses
simple forecasts of the series to remove the cyclical nature. Specifically, to produce a smoothed estimate
of

*Y*at time*t*, we use the fitted value from a regression of Y on 4 lagged values of Y back-shifted by two years (so 8 observation in quarterly data), and a constant. Specifically:
$$\widetilde{y}_t = \alpha_0 + \beta_1 y_{t-8} + + \beta_2 y_{t-9} + \beta_3 y_{t-10} + \beta_4 y_{t-11}$$

## An Example Using EViews.

Professor Hamilton provides some examples using employment
data, in the csv file employment.csv. Specifically,
the file contains quarterly non-farm payroll numbers, both seasonally adjusted
and non-seasonally adjusted between 1947 and 2016Q2.

We can open that file in EViews simply by dragging it to
EViews. The file doesn’t have a date
format that EViews understands, so we will manually restructure the page to
quarterly frequency with a start date of 1947Q1.

We then give the two time-series names, and create growth
rate series.

Having created the series we’re interested in, we’ll first
perform the HP filter on the seasonally adjusted series. We open the series,
click on Proc->Hodrick-Prescott Filter.
We enter names for the outputted trend and cycle series, and then click
OK.

Now we’ll replicate Hamilton’s method. We first need to regress the series against
four lags of itself shifted 8 periods back.
We do this using the Quick->Estimate Equation menu, then entering the
specification of NFP_LOG C NFP_LOG(-8
TO -11)

We can then view the residuals and fitted values, which
correspond to the cyclical and trend components from the View menu.

If we wanted to save those components, we could use the
Proc->Make Resids and Proc->Forecast menu items to produce the residuals
and fitted (forecasted) values.

We’ve written a quick EViews program that automates this
process for both the seasonally adjusted and non-seasonally adjusted data, and
replicates Figure 5 from Hamilton’s paper.
The program produces the following graphs, and the code is below:

'open data

wfopen .\employment.csv

'structure the data and rename
series

pagestruct(freq=q, start=1947)

d series01

rename series02 emp_sa

rename series03 emp_nsa

'calculate transforms of series

series nfp_log = 100*log(emp_sa)

series nsa_log = 100*log(emp_nsa)

'hp filter of employment
(seasonally adjusted)

nfp_log.hpf nfp_hptrend @ nfp_hpcycle

'hp filter of employment (non
seasonally adjusted)

nsa_log.hpf nsa_hptrend @ nsa_hpcycle

'estimate employment (seasonally
adjusted) regressed against constant and 4 lags of itself, offset by 8 periods.

equation eq1.ls nfp_log c nfp_log(-8 to -11)

'store resids as the cycle

eq1.makeresid nfp_cycle

'store fitted vals as the trend

eq1.fit nfp_trend

'estimate employment
(non-seasonally adjusted) regressed against constant and 4 lags of itself,
offset by 8 periods.

equation eq2.ls nsa_log c nsa_log(-8 to -11)

'store resids as the cycle

eq2.makeresid nsa_cycle

'store fitted vals as the trend

eq2.fit nsa_trend

'calculate 8 period differences

series nfp_base = nfp_log-nfp_log(-8)

series nsa_base = nsa_log-nsa_log(-8)

'display graphs of Hamilton's
method (replicate Figure 5)

freeze(g1) nfp_log.line

g1.addtext(t) Employment (seasonally adjusted)

call shade(g1)

freeze(g2) nsa_log.line

g2.addtext(t) Employment (not seasonally
adjusted)

call shade(g2)

group nfps nfp_cycle nfp_base

freeze(g3) nfps.line

g3.addtext(t) Cyclical components (SA)

g3.setelem(1) legend(Random Walk)

g3.setelem(2) legend(Regression)

g3.legend columns(1) position(4.67,0)

call shade(g3)

group nsas nsa_cycle nsa_base

freeze(g4) nsas.line

g4.addtext(t) Cyclical components (NSA)

g4.setelem(1) legend(Random Walk)

g4.setelem(2) legend(Regression)

g4.legend columns(1) position(4.67,0)

call shade(g4)

graph g5.merge g1 g2 g3 g4

g5.addtext(t) Figure 5 (Hamilton)

show g5

'display graphs of HP filter
results compared with Hamilton's

group nfp_cycles nfp_cycle nfp_hpcycle

freeze(g6) nfp_cycles.line

g6.addtext(t) Employment (seasonally adjusted)
Cycles

g6.setelem(1) legend(Hamilton Method)

g6.setelem(2) legend(HP Filter)

call shade(g6)

group nsa_cycles nsa_cycle nsa_hpcycle

freeze(g7) nsa_cycles.line

g7.addtext(t) Employment (Non-seasonally
adjusted) Cycles

g7.setelem(1) legend(Hamilton Method)

g7.setelem(2) legend(HP Filter)

call shade(g7)

graph g8.merge g6 g7

show g8

'subroutine to shade graphs using
Hamilton's dates (note these dates may differ slightly from the recession
shading add-in available in EViews).

'Also does some minor formatting.

subroutine shade(graph g)

g.draw(shade,
b) 1948q4 1949q4

g.draw(shade,
b) 1953q2 1954q2

g.draw(shade,
b) 1957q3 1958q2

g.draw(shade,
b) 1960q2 1961q1

g.draw(shade,
b) 1969q4 1970q4

g.draw(shade,
b) 1973q4 1975q1

g.draw(shade,
b) 1980q1 1980q3

g.draw(shade,
b) 1981q3 1982q4

g.draw(shade,
b) 1990q3 1991q1

g.draw(shade,
b) 2001q1 2001q4

g.draw(shade,
b) 2007q4 2009q2

g.datelabel
interval(year, 10, 0)

g.axis
minor

g.axis(b)
ticksout

g.options
-gridl

g.options
gridnone

endsub

Nice job! Thanks.

ReplyDeleteThe great Professor Giles!

DeleteThank you a lot.

ReplyDeleteExcuse me please, is this method good dealing with phase shifts?

ReplyDeleteI have one further question. How would I proceed, if I have monthly instead of quarterly data?

ReplyDeleteShift it by 24 periods rather than 8.

DeleteNice demonstration

ReplyDeleteI have one further question. How would I proceed, if I have annual data instead of quarterly data or monthly?

ReplyDeleteNice demonstration. But what if it is annual data? how many periods will we offset it and what is the appropriate lag? thanks

ReplyDeleteVery much depends on the study in question. I suggest reading Section 4.3 of Hamilton's paper.

DeleteHi!

ReplyDeleteWhere can I get the data?

http://econweb.ucsd.edu/~jhamilto/software.htm#HP

DeleteWonderful. How come the fitted value is not smooth like the HP trend?

ReplyDeleteThis a great post, I have applied both filters using this code, the cycles produced by Hamilton's regression filter tend to be longer and timing of the cycles is slightly different between the two methods

ReplyDeleteAfter implementing the manual procedure, I realized that the residuals and the fitted values (forecasted) have 11 missing data points by showing NA. Is that right please? Thank you.

ReplyDelete