How do I evaluate the suitability of a GARCH model?

$\begingroup$ To simplify, I only have two questions. The first one: GARCH is used to predict volatility. But, after we get the new return(or price) data, how well is the GARCH prediction? Is there any quantitative method to evaluate this? The second one: Once we decided to use GARCH(p,q), how do we choose the order p and q?(for example, if we choose (p,q)=(1,1), then why we do not choose(2,2) or whatever) $\endgroup$

$\begingroup$ Ok, so that's your point 1) and 3) I believe. Please edit your question and put the simplified version. $\endgroup$

2 Answers 2

Which model to choose from a pool of candidate models depends on what you want to do with it.

If you want to do forecasting, you should select a model that would be expected to deliver the most accurate forecasts. Akaike's information criterion (AIC) is known to asymptotically select a model (from a given pool) that will deliver the forecasts with the lowest mean squared error. Therefore, it is often used as a model selector in forecasting exercises, and justifiably so. AIC is easy to obtain from a fitted model (by hand or via a built-in function) , so it makes for a handy tool. An alternative to AIC could be the focused information criterion (FIC), which allows for much more flexibility in specifying what kind of loss function you wish to minimize (e.g. mean absolute error in place of mean squared error), but it also requires more involvement from the researcher; you need to derive the specific version of FIC based on your loss function before you can apply it.

If you want to recover the true model and the true model happens to be among the set of candidate models, use Bayesian information criterion (BIC). Asymptotically it will select the true model with probability=1. Since most of the time the true model is not in the pool (the reality is more complex than our models), BIC will tend to select a model that approximates the true model the most closely.

If you have yet another goal, a philosophically straightforward way to do model selection is to derive a version of FIC that reflects your goal (but again, it may be a challenging exercise on its own).

In all these cases, the candidate model pool has to be selected in the first place, and that is not a trivial task either. You may bring in subject-matter knowledge; probably you have an idea what type of models should describe your data generating process well. You may then explore a pool as large as is computationally feasible. However, bear in mind that information criteria are not entirely robust against overfitting. When you have a number of models with the same number of parameters, both AIC and BIC reduce to maximizing the likelihood, and that is not at all robust against overfitting. So in practice you may try a somehwta large pool but without going to extreme sizes.

The practice of seeking a model with statistically significant coefficients and well-behaved residuals as suggested in another answer is somehow tangential to the approach above. I am not sure the latter strategy optimizes any sensible loss function except for delivering a nice in-sample fit (and I mean: I am not sure). However, in sample we already know everything, we know each data point. The real goal is normally to generalize out of sample and either say something about the properties of the data generating process (e.g. test some hypotheses about it) or do forecasting. So then the approach above based on AIC, BIC and FIC makes sense. (Even so, recall that once model selection has been performed and you have ended up with a model of choice, you cannot trust the significance tests anymore because they are conditional on the model having been selected. The way of accounting for this conditioning properly is becoming an active research area in statistics.)

And then I have not touched upon regularized estimation and model averaging which are more advanced approaches that generalize the problem of model selection.

Also, you may find much more about all this at Cross Validated, a sister site in the Stack Exchange system.