Part V: Nonlinear and Nonparametric Models
We conclude with discussion and examples of more general models: parametric nonlinear models in Chapter 19 and then nonparametric models in Chapters 20–23. Parametric nonlinear models have some prespecified functional form with unknown parameters. Nonparametric models are characterized by having unknown functions without prespecified form as part of the model specification. Nonparametric refers to similarity with non-Bayesian nonparametric methods or to the fact that the parameters do not have the same sorts of interpretations as those in parametric models.
In Chapters 20 and 21 we discuss nonparamteric models, which may have a finite but arbitrarily large number of parameters allowing to approximate any function to any desired degree of accuracy. In Chapters 22 and 23 we discuss infinite dimensional generalizations where the prior is defined directly in infinite dimensional parameter space or function space. Actual computation is still made with finite dimensional presentation, but the number of parameters can increase as n grows.
The key defining property for a useful nonparametric Bayesian model is that the prior distribution for an unknown density has large support, which informally means that the prior can generate functions that are arbitrarily close to any function in a broad class. Consider the density estimation problem and let f ~ π, with π denoting a prior over the set of all densities on the real line. Our notion of large support requires that the prior π assigns positive probability to arbitrarily small neighborhoods of the true density f0 for a large class of true densities. If the prior assigns zero probability to a neighborhood of the true density f0, then the posterior will also assign zero probability to the neighborhood. Hence, if the true f0 is not in the support of the prior π, then one will obtain inconsistent estimates of the density and finite sample inferences may be misleading. It follows logically that, in the absence of substantial prior knowledge about the shape of the density, one should ideally choose the prior π so that all f0 (or perhaps a large subset discarding irregular densities) fall in the support of π.