M. Ivette Gomes1, Frederico Caeiro2, Lígia Henriques-Rodrigues3 and B.G. Manjunath4
1Universidade de Lisboa, FCUL, DEIO and CEAUL, Portugal
2Universidade Nova de Lisboa, FCT and CMA, Portugal
3Universidade de São Paulo, IME and CEAUL, Brazil
4Universidade de Lisboa, CEAUL, Portugal
AMS 2010 subject classification. Primary 62G32, 62E20; Secondary 65C05.
AMS 2000 subject classification. Primary 62G32, 62E20; Secondary 62G09, 62G30.
Let be a random sample from an underlying cumulative distribution function (CDF) . If we assume that is known, we can easily estimate the sampling distribution of any estimator of an unknown parameter through the use of a Monte Carlo simulation, described in the following algorithm:
If goes to infinity, something not achievable in practice, we should then get a perfect match to the theoretical calculation, if available, that is, the Monte Carlo error should disappear. But is usually unknown. How to proceed? The use of the bootstrap methodology is a possible way.
Bootstrapping (Efron, 1979) is essentially a computer-based and computer-intensive method for assigning measures of accuracy to sample estimates (see Efron and Tibshirani, 1994; Davison and Hinkley, 1997, among others). Concomitantly, this technique also allows estimation of the sampling distribution of almost any statistic using only very simple resampling methods, based on the observed value of the empirical distribution function, given by
We can replace in the previously sketched algorithm by , the empirical distribution function associated with the original observed data, , which puts mass on each of the , generating with replacement , in Step S1.1 of the algorithm in the preceding text, computing , , in Step S1.2, and using next such a sample in Step S2.
The main goal of this chapter is to enhance the role of the bootstrap methodology in the field of statistics of univariate extremes, where the bootstrap has been commonly used in the choice of the number of top order statistics or of the optimal sample fraction, , to be taken in the semiparametric estimation of a parameter of extreme events. For an asymptotically consistent choice of the threshold to use in the adaptive estimation of a positive extreme value index (EVI), , the primary parameter in statistics of extremes, we suggest and discuss a double-bootstrap algorithm. In such algorithm, apart from the classical Hill (1975) and peaks over random threshold (PORT)-Hill EVI estimators (Araújo Santos et al., 2006), we consider a class of minimum-variance reduced-bias (MVRB), the simplest one in Caeiroet al. (2005), and associated PORT-MVRB (Gomes et al., 2011a, 2013) EVI estimators. Other bootstrap methods for the choice of can be found in Hall (1990), Longin (1995), Caers et al. (1999), Draisma et al. (1999), Danielsson et al. (2001), and Gomes and Oliveira (2001), among others. For a recent comparison between the simple-bootstrap and the double-bootstrap methodology, see Caeiro and Gomes (2014b), where an improved version of Hall's bootstrap methodology was introduced.
After providing, in Section 6.2, a few technical details in the area of extreme value theory (EVT), related to the EVI estimators under consideration in this chapter, we shall briefly discuss, in Section 6.3, the main ideas behind the bootstrap methodology and optimal sample fraction estimation. In the lines of Gomes et al. (2011b–2012, 2015a), we propose an algorithm for the adaptive consistent estimation of a positive EVI, through the use of resampling computer-intensive methods. The Algorithm is described for the Hill EVI estimator and associated PORT-Hill, MVRB, and PORT-MVRB EVI estimators, but it can work similarly for the estimation of other parameters of extreme events, like a high quantile, the probability of exceedance, or the return period of a high level. The associated code in R language for the adaptive EVI estimation is available upon request. Section 6.4 is entirely dedicated to the application of the Algorithm to three simulated samples. Finally, in Section 6.5, we draw some overall conclusions.
The key results obtained by Fisher and Tippett (1928) on the possible limiting laws of the sample maxima, formalized by Gnedenko (1943), and used by Gumbel (1958) for applications of EVT in engineering subjects are some of the key tools that led to the way statistical EVT has been exploding in the last decades. In this chapter, we focus on the behavior of extreme values of a data set, dealing with maximum values and other top order statistics in a univariate framework, working thus in the field of statistics of extremes.
Let us assume that we have access to a random sample of independent, identically distributed, or possibly stationary and weakly dependent random variables from an underlying model , and let us denote by the sample of associated ascending order statistics. As usual, let us further assume that it is possible to linearly normalize the sequence of maximum values, , so that we get a nondegenerate limit. Then (Gnedenko, 1943), that limiting random variable has a CDF of the type of the extreme value distribution, given by
and is the so-called EVI, the primary parameter in statistics of extremes. We then say that is in the max-domain of attraction of , in (6.2), and use the notation . The EVI measures essentially the weight of the right tail function, . If , the right tail is short and light, since has compulsory a finite right end point, that is, is finite. If , the right tail is heavy and of a negative polynomial type, and has an infinite right end point. A positive EVI is also often called tail index. If , the right tail is of an exponential type, and the right end point can then be either finite or infinite.
Slightly more restrictively than the full max-domain of attraction of the extreme value distribution, we now consider a positive EVI, that is, we work with heavy-tailed models in . Heavy-tailed models appear often in practice in fields like bibliometrics, biostatistics, finance, insurance, and telecommunications. Power laws, such as the Pareto distribution and Zipf's law, have been observed a few decades ago in some important phenomena in economics and biology and have seriously attracted scientists in recent years. As usual, we shall further use the notations for the generalized inverse function of and for the class of regularly varying functions at infinity with an index of regular variation , that is, positive Borel measurable functions such that , as , for all (see Bingham et al., 1987, for details on regular variation). Let us further use the notation for the tail quantile function. For heavy-tailed models we have the validity of the following first-order conditions:
The first necessary and sufficient condition in the preceding text, related to the right tail function behavior, was proved by Gnedenko (1943), and the second one, related to the tail quantile function behavior, was proved by de Haan (1984).
For these heavy-tailed models, and given a sample , the classical EVI estimators are Hill estimators (Hill, 1975), with the functional expression
They are thus the average of the log-excesses, , above the random level or threshold . To have consistency of Hill EVI estimators, we need to have , and such a random threshold needs further to be an intermediate order statistic, that is, we need to have
if we want to have consistent EVI estimation in the whole . Indeed, under any of the first-order frameworks in (6.3), the log-excesses, , are approximately the order statistics of an exponential sample of size , with mean value , hence the reason for the EVI estimators in (6.4).
Under adequate second-order conditions that rule the rate of convergence in any of the first-order conditions in (6.3), Hill estimators, , have usually a high asymptotic bias, and recently, several authors have considered different ways of reducing bias (see the overviews in Gomes et al., 2007b, Chapter 6 of Reiss and Thomas, 2007; Gomes et al., 2008a; Beirlant et al., 2012; Gomes and Guillou, 2015). A simple class of MVRB EVI estimators is the class studied in Caeiro et al. (2005), to be introduced in Section 6.2.2. These MVRB EVI estimators depend on the adequate estimation of second-order parameters, and the kind of second-order parameter estimation that enables the building of MVRB EVI estimators, that is, EVI estimators that outperform the Hill estimator for all , is sketched in Sections 6.2.1 and 6.2.2.
Both Hill and MVRB EVI estimators are invariant to changes in scale but not invariant to changes in location. And particularly the Hill EVI estimators can suffer drastic changes when we induce an arbitrary shift in the data, as can be seen in Figure 6.1.
Indeed, even if a Hill plot (a function of vs ) looks stable, as happens in Figure 6.1, with the sample path, where data, , , come from a unit standard Pareto CDF, , for ), we easily come to the so-called Hill horror plots, a terminology used in Resnick (1997), when we induce a shift to the data. This can be seen also in Figure 6.1 (look now at ), where we present the Hill plot associated with the shifted sample , from the CDF , now for . This led Araújo Santos et al. (2006) to introduce the so-called PORT methodology, to be sketched in Section 6.2.3. The asymptotic behavior of the EVI estimators under consideration is discussed in Section 6.2.4.
For consistent semiparametric EVI estimation, in the whole , we have already noticed that we merely need to work with adequate functionals, dependent on an intermediate tuning or control parameter , the number of top order statistics involved in the estimation, that is, (6.5) should hold. To obtain full information on the nondegenerate asymptotic behavior of semiparametric EVI estimators, we often need to further assume a second-order condition, ruling the rate of convergence in any of the first-order conditions, in (6.3). It is often assumed that there exists a function , such that
Then, we have . Moreover, if the limit in the left-hand side of (6.6) exists, we can choose so that such a limit is compulsory equal to the previously defined function (Geluk and de Haan, 1987).
Whenever dealing with reduced-bias estimators of parameters of extreme events, and essentially due to technical reasons, it is common to slightly restrict the domain of attraction, , and to consider a Pareto-type class of models, assuming that, with , , , and as ,
The class in (6.7) is however a wide class of models that contains most of the heavy-tailed parents useful in applications, like the Fréchet, the generalized Pareto, and the Student-, with degrees of freedom. For Fréchet parents, we get and in (6.7). For a generalized Pareto distribution, , with given in (6.2), we get and . For Student- parents, we get and . For further details and an explicit expression of as a function of , see Caeiro and Gomes (2008), among others. Note that the validity of (6.6) with is equivalent to (6.7). To obtain information on the bias of MVRB EVI estimators, it is even common to slightly restrict the class of models in (6.7), further assuming the following third-order condition:
as , with . All the aforementioned models still belong to this class. Slightly more generally, we could have assumed a general third-order condition, ruling now the rate of convergence in the second-order condition in (6.6), which guarantees that, for all ,
where must then be in . Equation (6.8) is equivalent to equation (6.9) with . Further details on the topic can be found in de Haan and Ferreira (2006).
Provided that (6.5) and (6.6) hold, Hill EVI estimators, , have usually a high asymptotic bias. The adequate accommodation of this bias has recently been extensively addressed. Among the pioneering papers, we mention Peng (1998), Beirlant et al. (1999), Feuerverger and Hall (1999), and Gomes et al. (2000). In these papers, authors are led to reduced-bias EVI estimators, with asymptotic variances larger than or equal to , where is the aforementioned “shape” second-order parameter in (6.6). Recently, as sketched in Section 6.2.2, Caeiro et al. (2005) and Gomes et al. (2007a, 2008c) have been able to reduce the bias without increasing the asymptotic variance, kept at , just as happens with the Hill EVI estimator.
To reduce bias, keeping the asymptotic variance at the same level, we merely need to use an adequate “external” and a bit more than consistent estimation of the pair of second-order parameters, , in (6.7). The MVRB EVI estimators outperform the classical Hill EVI estimators for all , and among them, we now consider the simplest class by Caeiro et al. (2005), used for value-at-risk (VaR) estimation by Gomes and Pestana (2007b). Such a class, denoted by , has the functional form
where is an adequate consistent estimator of , with and based on a number of top order statistics usually of a higher order than the number of top order statistics used in the EVI estimation, as explained in Sections 6.2.1 and 6.2.2. For different algorithms for the estimation of , see Gomes and Pestana (2007a,b).
We consider the most commonly used -estimators, the ones studied by Fraga Alves et al. (2003), briefly introduced in the sequel. Given the sample , the -estimators by Fraga Alves et al. (2003) are dependent on the statistics
defined for any tuning parameter and where
Under mild restrictions on , that is, if (6.5) holds and , with the function in (6.7), the statistics in (6.11) converge toward , independently of the tuning parameter , and we can consequently consider the class of admissible -estimators:
Under adequate general conditions, and for an appropriate tuning parameter , the -estimators in (6.12) show highly stable sample paths as functions of , the number of top order statistics used, for a range of large -values. Again, it is sensible to advise practitioners not to choose blindly the value of in (6.12). Sample paths of , as functions of , for a few values of , should be drawn, in order to elect the value of , which provides higher stability for large , by means of any stability criterion. For the most common stability criterion, see Gomes and Pestana (2007b) and Remark 6.6. The value , considered in the description of the Algorithm in Section 6.3.2, has revealed to be the most adequate choice whenever we are in the region , a common region in applications, and the region where bias reduction is indeed needed. Distributional properties of the estimators in (6.12) can be found in Fraga Alves et al. (2003). Interesting alternative classes of -estimators have recently been introduced by Goegebeur et al. (2008, 2010), Ciuperca and Mercadier (2010), and Caeiro and Gomes (2014a, 2015).
For the estimation of the scale second-order parameter , on the basis of
we shall consider the estimator in Gomes and Martins (2002):
dependent on an adequate -estimator, . It has been advised the computation of these second-order parameter estimators at a -value given by
The estimator , to be plugged in (6.13), is thus , with and given in (6.12) and (6.14), respectively.
Details on the distributional behavior of the estimator in (6.13) can be found in Gomes and Martins (2002) and more recently in Gomes et al. (2008c) and Caeiro et al. (2009). Again, consistency is achieved for models in (6.7) and -values such that (6.5) holds and , as . Alternative estimators of can be found in Caeiro and Gomes (2006) and Gomes et al. (2010). Due to the fact that and , with , , and given in (6.12)–(6.14), respectively, depending on , we often use the notation . But when we work with only, as happens in Section 6.3.2, we shall not use the subscript . Note however that the Algorithm in Section 3.2 can also be used for another fixed choice of , as well as for a data-driven choice of provided by any of the algorithms in Gomes and Pestana (2007a,b), among others.
The estimators in (6.4) and (6.10) are scale invariant but not location invariant. In order to achieve location invariance for a class of modified Hill EVI estimators and adequate properties for VaR estimators, Araújo Santos et al. (2006) introduced the so-called PORT methodology. The estimators are then functionals of a sample of excesses over a random level , , that is, functionals of the sample
Generally, we can have , for any (the random level is an empirical quantile). If the underlying model has a finite left end point, , we can also use (the random level can then be the minimum).
If we think, for instance, on Hill EVI estimators, in (6.4), the new classes of PORT-Hill EVI estimators, theoretically studied in Araújo Santos et al. (2006), and for finite samples in Gomes et al. (2008b), are given by
Similarly, if we think on the MVRB EVI estimators, in (6.10), the new classes of PORT-MVRB EVI estimators, studied for finite samples in Gomes et al. (2011a, 2013), are given by
with in (6.16), and any adequate estimator of , the vector of second-order parameters associated with the shifted model, based on the sample , in (6.15).
These PORT EVI estimators are thus dependent on a tuning parameter , 0⩽q<1, that makes them highly flexible. Moreover, they are invariant to changes in both location and scale. Just as in Gomes et al. (2013, 2015a), we shall further include in the algorithm the value , so that with , , , and , given in (6.4), (6.10), (6.16), and (6.17), respectively, we can consider that and for (with the notations , , so that , , ).
Further applications of the PORT methodology can be found in Henriques-Rodrigues et al. (2014, 2015), Caeiro et al. (2016) and Gomes et al. (2016), among others.
The Hill estimator reveals usually a high asymptotic bias. Indeed, from the results of de Haan and Peng (1998), and with denoting a normal random variable with mean value and variance , there exists such that
where the bias under condition (6.8) can be very large, moderate, or small, going, respectively, to , a nonnull constant, or 0, as . This nonnull asymptotic bias, together with a rate of convergence of the order of , leads to sample paths with a high variance for small , a high bias for large , and a very sharp mean square error (MSE) pattern, as a function of . Under the same conditions as before, is asymptotically normal with variance also equal to but with a null mean value. Indeed, under the validity of the aforementioned third-order condition in (6.8), related to Pareto-type class of models, we can adequately estimate the vector of second-order parameters so that outperforms for all . Indeed, and for an adequate , computed by Caeiro et al. (2009), we can write
We can further summarize the aforementioned results in the following theorem.
For the asymptotic behavior of the PORT-Hill EVI estimators, we refer to Araújo Santos et al. (2006). The full asymptotic behavior of the PORT-MVRB EVI estimators is still under development. It is known that the rate of convergence and asymptotic variance do not change. There are however big changes in the bias but for adequate -values the PORT-MVRB EVI estimators are indeed MVRB EVI estimators. Contrarily to what has been done by Gomes et al. (2015a), we shall thus consider for them the same double-bootstrap Algorithm we used for the MVRB EVI estimation.
The use of bootstrap resampling methodologies has revealed to be promising in the choice of the nuisance tuning or control parameter or equivalently of the optimal sample fraction, , in the semiparametric estimation of any parameter of extreme events. If we ask how to choose the tuning parameter in the EVI estimation, either through or or or , , generally denoted , we usually consider the estimation of
To obtain estimates of , one can use the so-called double-bootstrap method based on two related bootstrap samples of size and . Such a method is applied to an adequate auxiliary statistic like
which tends to the well-known value zero and has an asymptotic behavior similar to the one of (see Gomes and Oliveira, 2001, among others, for the estimation through and Gomes et al., 2012, for the estimation through ). See also Gomes et al. (2015a,b) and Section 6.3.2.
On the basis of (6.20) and (6.21), and with AMSE standing for “asymptotic MSE,” the sum of the asymptotic variance and the squared dominant component of the bias, we get
with defined in (6.22). See Theorem 1 of Draisma et al. (1999), for a proof of this result, in the case of . The proof is similar for the cases of , as already mentioned by Gomes et al. (2012). Things work more intricately for the PORT-MVRB EVI estimators, and as mentioned in the preceding text, we shall consider an algorithm similar to the one devised for the MVRB EVI estimators in case we are working with , , since we are interested in the possible specific value of that makes these PORT estimators MVRB EVI estimators. The bootstrap methodology enables us to estimate , in (6.22), in a way similar to the one used for the classical EVI estimators, on the basis of a consistent estimator of , in (6.24), and now through the use of an auxiliary statistic like the one in (6.23), a method detailed in Gomes et al. (2011b–2012) for the MVRB EVI estimation. For the sake of simplicity, we shall next describe the methodology for , but similar formulas work for provided that we replace by , . Indeed, under the aforementioned third-order framework in (6.8),
with asymptotically standard normal.
Consequently, denoting , we have
How does the resampling methodology then work? Given the sample from an unknown model , and the functional in (6.23), , , consider for any , , the bootstrap sample
from , in (6.1), the empirical distribution function associated with the available random sample, .
Next, associate with the bootstrap sample the corresponding bootstrap auxiliary statistic, , . Then, with ,
Consequently, for another sample size , and for every ,
It is then enough to choose , in order to have independence of . If we consider , that is, , we have
On the basis of (6.26), we are now able to consistently estimate and next through (6.25), on the basis of any estimate of the second-order parameter . With denoting the sample counterpart of and , an adequate -estimate, we thus have the estimate
with
The adaptive estimate of is then given by
In the following Algorithm we include the Hill, the MVRB, the PORT-Hill and the PORT-MVRB EVI estimators in the overall selection.
To enhance the importance of the PORT-Hill and PORT-MVRB EVI estimation in the field of finance, we refer to Gomes and Pestana (2007b) and Gomes et al. (2013), where, respectively, the MVRB and the PORT-MVRB EVI estimation have been applied to log returns associated with a few sets of financial data. Due to the specificity of such real data sets and to the fact that log returns have often been modeled by a Student- or its skewed versions (see Jones and Faddy, 2003, among others), we have simulated a random sample of size , from a Student's model with degrees of freedom ( and ). Due to the specificity of the data (infinite left end point), we have considered for both the PORT-Hill and the PORT-MVRB EVI estimation -values from 0.15 to 1, with step 0.05. When , we elect the Hill or the MVRB EVI estimates. If , the PORT methodology is elected. We have further considered , with from 0.950 to 0.995, with step 0.0025, and .
Figure 6.3 is related to the Student- generated sample, and we there present as an illustration of the obtained results the PORT-Hill/Hill and PORT-MVRB/MVRB EVI estimates (a), the -estimates (b), and the RMSE estimates (c). The notation PORT •/• means that we are playing with both the PORT-• and the • EVI estimators. We have however been led to a PORT estimator, that is, to -estimates smaller than 1.
Research partially supported by national funds through FCT—Fundação para a Ciência e a Tecnologia, projects UID/MAT/UI0006/2013 (CEA/UL) and UID/MAT/0297/2013 (CMA/UNL) and postdoc grants SFRH/BPD/77319/2011 and SFRH/BPD/72184/2010.