Table of Contents
 Basic Principles
 Comprehensiveness: What Does SpecEval Do?
 Flexibility in Practice
 What’s Next?
 Footnotes
Basic Principles
The idea behind SpecEval is simple: to do model development effectively – especially in time constrained environment – one should have a tool that can quickly produce and summarize information about particular model. Such tool should satisfy three key requirements: It should be very easy to use, so that its use does not introduce additional costs into the model development process.
 It should be comprehensive in the sense that it includes all relevant information one would like to have when evaluating particular model.
 It should be flexible so that user can easily change what information is included in particular situations. Flexibility is a necessary counterpart of comprehensiveness so that one avoids congestion.
Comprehensiveness: What Does SpecEval Do?
So what does SpecEval addin do? In broad terms, it produces tables and graphs that provide information about the model, and especially its behavior. Note here that discussing the set of possible outputs (listed in the table below) is not in the scope of this blog post since most functionality will be illustrated in the blog posts to follow. Instead the table should highlight that the addin is indeed comprehensive from a model development perspective.^{1}Object Name  Description 
Estimation output table  Adjusted regression output table 
Coefficient stability graph  Graph with recursive equation coefficients 
Model stability graph  Graph with recursive lag orders 
Performance metrics tables  Table with values of forecast performance metrics 
Performance metrics tables (multiple specifications)  Table with values of forecast performance metrics for given metric for all specifications 
Forecast summary graph  Graph with all recursive forecasts with given horizons 
Subsample forecast graph  Graph with forecast for given subsample 
Subsample forecast decomposition graph  Graph with decomposition of subsample forecast 
Forecast bias graph  Scatter plot of forecast and actual values for given forecast horizon (MinzerZarnowitz plot) 
Individual conditional scenario forecast graph (level)  Graph with forecast for single scenario and specification 
Individual conditional scenario forecast graph (transformation)  Graph with transformation of forecast for single scenario and specification 
All conditional scenario forecast graph  Graph with forecasts for all scenarios for single specification 
Multiple specification conditional scenario forecast graph  Graph with forecasts for single scenario for multiple specifications 
Shock response graphs  Graphs with response to shock to individual independent variable/regressor 












Flexibility in Practice
The third basic principle – flexibility – is in practice embodied in the ability of the user to adjust the processes or the outputs via addin options. There are altogether almost 40 user settings – all listed and explained in the addin documentation  which can be divided into several categories.First, general options focus on which of the inbuilt functionality is going to be performed and on which objects/specifications. Next, there is a group of options that allows customization of the outputs, such as specification of horizons for tables and/or graphs, transformations used in graphs, or additional series to be included in graphs. Third group of options allows for some basic customization of the forecasting processes. For example, one can choose between insample and outofsample forecasting, or one can specify additional equations/identities to be treated as part of the forecasting model.^{2} These are just two examples in which the forecasting process can be customized.
Final two groups focus on control of samples used in the various procedures and on customization of storage settings. The former includes for example an option to manually specify sample boundaries for the backtesting procedures, or for the conditional scenario forecasts. The latter then allows the user to determine which objects will be kept in the workfile after the execution and under what names or aliases.
What's Next
Future blog posts in this series will focus on illustrating both the use of the addin, highlighting the ease of use and flexibility, and on the outputs. Each will follow a particular application, always focusing on a particular feature(s) of the addin. First in the series will provide overview of basics of using the addin, highlighting the key outputs and the customization of the process and the outputs. Second in the series will then stress the ability  and power  of using transformations in model development. Third post will focus on creating unconditional forecasts, while the last post will conclude with a brief look at recursive model structures.Footnotes
^{1. Of course, comprehensiveness is more a goal rather than a state in that there will always be additional functionalities that could/should be included. See model development list on the addin GitHub site for what additional functionality is on the roadmap, but feel free to also make suggestions there. Also, the addin is comprehensive in terms of its focus, which is forecasting behavior of a given model – as opposed to econometric characteristics of the model. This means that currently the addin does not include any information in the form of outputs of econometric tests.↩}^{2. By historical forecasts I mean conditional forecasts, which are potentially multistep and dynamic, and/or recursive.↩}
^{3. Note that these two features – insample forecasting and inclusion of multiple equations in the forecasting model – are possible thanks to inbuilt EViews functionality and hard to replicate in other statistical programs. The former is thanks to the separation between estimation and forecasting samples, the latter thanks to flexible model objects.↩}
No comments:
Post a Comment