10
Rank‐Based Shrinkage Estimation
This chapter introduces the R‐estimates and provides a comparative study of ridge regression estimator (RRE), least absolute shrinkage and selection operator (LASSO), preliminary test estimator (PTE) and the Stein‐type estimator based on the theory of rank‐based statistics and the nonorthogonality design matrix of a given linear model.
10.1 Introduction
It is well known that the usual rank estimators (REs) are robust in the linear regression models, asymptotically unbiased with minimum variance. But, the data analyst may point out some deficiency with the R‐estimators when one considers the “prediction accuracy” and “interpretation.” To overcome these concerns, we propose the rank‐based least absolute shrinkage and selection operator (RLASSO) estimator. It defines a continuous shrinking operation that can produce coefficients that are exactly “zero” and competitive with the rank‐based “subset selection” and RRE, retaining the good properties of both the R‐estimators. RLASSO simultaneously estimates and selects the coefficients of a given linear regression model.
However, there are rank‐based PTEs and Stein‐type estimators (see Saleh 2006; Jureckova and Sen 1996, and Puri and Sen 1986). These R‐estimators provide estimators which shrink toward the target value and do not select coefficients for appropriate prediction and interpretations. Hoerl and Kennard (1970) introduced ridge regression based on the Tikhonov (1963) regularization, and Tibshirani (1996) introduced the LASSO estimators in a parametric formulation. The methodology is minimization of least squares objective function subject to 
‐ and 
‐penalty restrictions. However, the 
penalty does not produce a sparse solution, but the 
‐penalty does.
This chapter points to the useful aspects of RLASSO and the rank‐based ridge regression R‐estimators as well as the limitations. Conclusions are based on asymptotic 
‐risk lower bound of RLASSO with the actual asymptotic 
risk of other R‐estimators.
10.2 Linear Model and Rank Estimation
Consider the multiple linear model,
where 
, 
is the 
matrix of real numbers, 
is the intercept parameter, and 
is the 
‐vector of regression parameters. We assume that:
- Errors
are independently and identically distributed (i.i.d.) random variables with (unknown) cumulative distributional function (c.d.f.) 
having absolutely continuous probability density function (p.d.f.) 
with finite and nonzero Fisher information
(10.2)
- For the definition of the linear rank statistics, we consider the score generating function
which is assumed to be nonconstant, non‐decreasing, and square integrable on 
so that
(10.3)
The scores are defined in either of the following ways:
for
, where 
are order statistics from a sample of size 
from 
. - Let
(10.4)where
is the 
th row of 
and 
, 
. We assume that
For the R‐estimation of the parameter 
, define for 
the rank of 
among 
by 
. Then for each 
, consider the set of scores 
and define the vector of linear rank statistics
Since 
are translation invariant, there is no need of adjustment for the intercept parameter 
.
If we set 
for 
, then the unrestricted RE is defined by any central point of the set
Let the RE be as 
. Then, using the uniform asymptotic linearity of Jureckova (1971),
for any 
and 
. Then, it is well‐known that
where
and
Similarly, when 
, the model reduces to
Accordingly, for the R‐estimation of 
, in this case, we define rank of 
among
as 
. Then, for each 
, consider the set of scores 
and define the vector of linear rank statistics
where
Then, define the restricted RE of 
as
which satisfy the following equality
where 
, for any 
and 
. Then, it follows that
We are basically interested in the R‐estimation of 
when it is suspected the sparsity condition 
may hold. Under the given setup, the RE is written as
where 
and 
are 
and 
vectors, respectively; and if 
is satisfied, then the restricted RE of 
is
where 
is defined by (10.13).
In this section, we are interested in the study of some shrinkage estimators stemming from 
and 
. In order to look at the performance characteristics of several R‐estimators, we use the two components asymptotic weighted 
‐risk function
where 
is any estimator of the form 
of 
.
Note that 
so that the marginal distribution of 
and 
are given by
Hence, asymptotic distributional weighted 
risk of 
is given by
For the test of sparsity 
, we define the aligned rank statistic
and 
. Further, it is shown (see, Puri and Sen 1986) that
It is easy to see that under 
, 
has asymptotically the chi‐square distribution with 
degrees of freedom (DF) and under the sequence of local alternatives 
defined by
For a suitable estimator 
of 
, we denote by
where we assume that 
is non‐degenerate.
One may find the asymptotic distributional weighted 
risks are
where
Our focus in this chapter is the comparative study of performance properties of these rank‐based penalty R‐estimators and PTEs and Stein‐type R‐estimators. We refer to Saleh (2006) for the comparative study of the PTEs and Stein‐type estimators. We extend the study to include penalty estimators which have not been done yet.
Now, we recall several results that form the asymptotic distribution of
For the RE given by 
, we see under regularity condition and Eq. (10.22),
where 
.
As a consequence, the asymptotic marginal distribution of 
(
) under local alternatives
is given by
where 
is 
th diagonal of 
.
10.2.1 Penalty R‐Estimators
In this section, we consider three basic penalty estimators, namely:
- The hard threshold estimator (HTE) (Donoho and Johnstone 1994),
- The LASSO by Tibshirani (1996),
- The RRE by Hoerl and Kennard (1970).
Motivated by the idea that only few regression coefficients contribute signal, we consider threshold rules that retain only observed data that exceed a multiple of the noise level. Accordingly, we consider the “subset selection” rule given by Donoho and Johnstone (1994) known as the “hard threshold” rule as given by
where 
is the 
th element of 
and 
is an indicator function of the set 
. The quantity 
is called the threshold parameter. The components of 
are kept as 
if they are significant and zero, otherwise. It is apparent that each component of 
is a PTE of the predictor concerned. The components of 
are PTEs and discrete variables and lose some optimality properties. Hence, one may define a continuous version of (10.29) based on marginal distribution of 
(
).
In accordance with the principle of the PTE approach (see Saleh 2006), we define the Stein‐type estimator as the continuous version of PTE based on the marginal distribution of
given by
See Saleh (2006, p. 83) for more details.
Another continuous version proposed by Tibshirani (1996) and Donoho and Johnstone (1994) is called the LASSO. In order to develop LASSO for our case, we propose the following modified R LASSO (MRL) given by
where for 
,
The estimator 
defines a continuous shrinkage operation that produces a sparse solution which may be derived as follows:
One may show that 
is the solution of the equation
Thus, the 
th component of (10.32) equals
Now, consider three cases, 
and 0. We can show that the asymptotic distributional expressions for 
is given by
Finally, we consider the R ridge regression estimator (RRRE) of 
. They are obtained using marginal distributions of 
, 
, as
to accommodate sparsity condition; see Tibshirani (1996) on the summary of properties discussed earlier.
Our problem is to compare the performance characteristics of these penalty estimators with that of the Stein‐type and preliminary test R‐estimators (PTREs) with respect to asymptotic distributional mean squared error criterion. We present the PTREs and Stein‐type R‐estimators in the next section.
10.2.2 PTREs and Stein‐type R‐Estimators
For the model (10.1), if we suspect sparsity condition, i.e. 
, then the restricted R‐estimator of 
is 
. For the test of the null‐hypothesis 
vs. 
, the rank statistic for the test of 
is given by
where 
, 
, and
It is well known that under the model (10.1) and assumptions (10.2)–(10.4) as 
, 
follows the chi‐squared distribution with 
DF Then, we define the PTE of 
as
where 
is the indicator function of the set 
.
Similarly, we define the Stein‐type R‐estimator as
Finally, the positive rule Stein‐type estimator is given by
10.3 Asymptotic Distributional Bias and 
Risk of the R‐Estimators
First, we consider the asymptotic distribution bias (ADB) and ADR of the penalty estimators.
10.3.1 Hard Threshold Estimators (Subset Selection)
It is easy to see that
where 
is the c.d.f. of a noncentral chi‐square distribution with 
DF and noncentrality parameter, 
.
The ADR of 
is given by
Since
for 
. Hence,
Note that (10.43) is free of the threshold parameter, 
.
Thus, we have Lemma 10.1, which gives the asymptotic distributional upper bound (ADUB) of 
as
If we have a sparse solution with 
coefficients 
(
) and 
zero coefficients, then
Then, the ADUB of the weighted 
‐risk is given by
is independent of the threshold parameter, 
.
10.3.2 Rank‐based LASSO
The ADB and ADR of 
are given by
and
where
Hence, the following Lemma 10.2 gives the ADUB of 
‐risk of 
.
As for the sparse solution, the weighted 
‐risk upper bound are given by
independent of 
.
Next, we consider “asymptotic oracle for orthogonal linear projection” (AOOLP) in the following section.
10.3.3 Multivariate Normal Decision Theory and Oracles for Diagonal Linear Projection
Consider the following problem in multivariate normal decision theory. We are given the least‐squares estimator (LSE) of 
, namely, 
according to
where 
is the marginal variance of 
, 
, and noise level and 
are the object of interest. We measure the quality of the estimator based on 
‐loss and define the risk as
If there is a sparse solution, then use the (10.18) formulation. We consider a family of diagonal linear projections,
Such estimators “keep” or “kill” the coordinate. The ideal diagonal coefficients, 
are in this case 
. These coefficients estimate those 
's which are larger than the noise level 
, yielding the asymptotic lower bound on the risk as
As a special case of (10.57), we obtain
In general, the risk 
cannot be attained for all 
by any estimator, linear or nonlinear. However, for the sparse case, if 
is the number of nonzero coefficients, 
and 
is the number of zero coefficients, then (10.58) reduces to the lower bound given by
Consequently, the weighted 
‐risk lower bound is given by
10.4 Comparison of Estimators
In this section, we compare various estimators with respect to the unrestricted R‐estimator (URE), in terms of relative weighted 
‐risk efficiency (RWRE).
10.4.1 Comparison of RE with Restricted RE
In this case, the RWRE of restricted R‐estimator versus RE is given by
where 
. The 
is a decreasing function of 
. So,
In order to compute 
, we need to find 
, 
, and 
. These are obtained by generating explanatory variables using the following equation following McDonald and Galarneau (1975),
where 
are independent 
pseudorandom numbers and 
is the correlation between any two explanatory variables. In this study, we take 
, and 0.9, which shows the variables are lightly collinear and severely collinear. In our case, we chose 
and various 
. The resulting output is then used to compute 
.
10.4.2 Comparison of RE with PTRE
Here, the RWRE expression for PTRE vs. RE is given by
where
Then, the PTRE outperforms the RE for
Otherwise, RE outperforms the PTRE in the interval 
. We may mention that 
is a decreasing function of 
with a maximum at 
, then it decreases crossing the 1‐line to a minimum at 
with a value 
, and then increases toward 1‐line.
The RWRE expression for PTRE vs. RE belongs to the interval
where 
depends on the size 
and given by
The quantity 
is the value 
at which the RWRE value is minimum.
10.4.3 Comparison of RE with SRE and PRSRE
To express the RWRE of SRE and PRSRE, we assume always that 
. We have then
It is a decreasing function of 
. At 
, its value is 
and when 
, its value goes to 1. Hence, for 
,
Also,
So that,
We also provide a graphical representation (Figure 10.1) of RWRE of the estimators for 
and 
.
Figure 10.1 RWRE for the restricted, preliminary test, Stein‐type, and its positive‐rule R‐estimators.
10.4.4 Comparison of RE and Restricted RE with RRRE
First, we consider the weighted 
‐risk difference of RE and RRRE given by
Hence, RRRE outperforms the RE uniformly. Similarly, for the restricted RE and RRRE, the weighted 
‐risk difference is given by
If 
, then (10.68) is negative. The restricted RE outperforms RRRE at this point. Solving the equation
we get
If 
, then the restricted RE outperforms the RRRE, and if 
, RRRE performs better than the restricted RE. Thus, neither restricted RE nor RRRE outperforms the other uniformly.
In addition, the RWRE of RRRE versus RE equals
which is a decreasing function of 
with maximum 
at 
and minimum 1 as 
. So,
10.4.5 Comparison of RRRE with PTRE, SRE, and PRSRE
Here, the weighted 
‐risk difference of PTRE and RRRE is given by
Since the first term is a decreasing function of 
with a maximum value 
at 
and tends to 0 as 
. The second function in brackets is also decreasing in 
with maximum 
at 
which is less than 
, and the function tends to 0 as 
. Hence, (10.72) is nonnegative for 
. Hence, the RRRE uniformly performs better than the PTRE.
Similarly, we show the RRE uniformly performs better than the SRE, i.e. the weighted 
risk of 
and 
is given by
The weighted 
‐risk difference of SRE and RRRE is given by
Since the first function decreases with a maximum value 
at 
, the second function decreases with a maximum value 
and tends to 0 as 
. Hence, the two functions are one below the other and the difference is nonnegative for 
.
Next, we show that the weighted 
risk (W
R) of the two estimators may be ordered as
Note that
where 
is defined by Eq. (9.70).
Thus, we find that the W
R difference is given by
Hence, the RRE uniformly performs better than the PRSRE.
10.4.6 Comparison of RLASSO with RE and Restricted RE
First, note that if 
coefficients 
and 
coefficients are zero in a sparse solution, the lower bound of the weighted 
risk is given by 
. Thereby, we compare all estimators relative to this quantity. Hence, the W
R difference between RE and RLASSO is given by
Hence, if 
, the RLASSO performs better than the RE; while if 
, the RE performs better than the RLASSO. Consequently, neither the RE nor the RLASSO performs better than the other uniformly.
Next, we compare the restricted RE and RLASSO. In this case, the W
R difference is given by
Hence, the RRE uniformly performs better than the RLASSO. If 
, RLASSO and RRE are 
‐risk equivalent. If the RE estimators are independent, then 
. Hence, RLASSO satisfies the oracle properties.
10.4.7 Comparison of RLASSO with PTRE, SRE, and PRSRE
We first consider the PTRE versus RLASSO. In this case, the W
R difference is given by
Hence, the RLASSO outperforms the PTRE when 
. But, when 
, then the RLASSO outperforms the PTRE for
Otherwise, PTRE outperforms the modified RLASSO estimator. Hence, neither PTRE nor the modified RLASSO estimator outperforms the other uniformly.
Next, we consider SRE and PRSRE versus the RLASSO. In these two cases, we have the W
R differences given by
and from (10.74),
where 
is given by (9.70). Hence, the modified RLASSO estimator outperforms the SRE as well as the PRSRE in the interval
Thus, neither SRE nor the PRSRE outperforms the modified RLASSO estimator uniformly.
10.4.8 Comparison of Modified RLASSO with RRRE
Here, the weighted 
‐risk difference is given by
Hence, the RRRE outperforms the modified RLASSO estimator, uniformly.
10.5 Summary and Concluding Remarks
In this section, we discuss the contents of Tables 10.1–10.10 presented as confirmatory evidence of the theoretical findings of the estimators.
Table 10.1 RWRE for the estimators for 
and 
.
| RRE | PTRE | ||||||||||||
 |
 |
||||||||||||
 |
RE | 0.1 | 0.2 | 0.8 | 0.9 | MRLASSO |  |
0.2 |  |
SRE | PRSRE | RRRE | |
| 0 | 1.0000 | 3.7179 | 3.9415 | 5.0593 | 5.1990 | 3.3333 | 1.9787 | 1.7965 | 1.6565 | 2.0000 | 2.3149 | 3.3333 | |
| 0.1 | 1.0000 | 3.5845 | 3.7919 | 4.8155 | 4.9419 | 3.2258 | 1.9229 | 1.7512 | 1.6194 | 1.9721 | 2.2553 | 3.2273 | |
| 0.5 | 1.0000 | 3.1347 | 3.2920 | 4.0374 | 4.1259 | 2.8571 | 1.7335 | 1.5970 | 1.4923 | 1.8733 | 2.0602 | 2.8846 | |
| 1 | 1.0000 | 2.7097 | 2.8265 | 3.3591 | 3.4201 | 2.5000 | 1.5541 | 1.4499 | 1.3703 | 1.7725 | 1.8843 | 2.5806 | |
 |
1.62 | 1.0000 | 2.3218 | 2.4070 | 2.7829 | 2.8246 | 2.1662 | 1.3920 | 1.3164 | 1.2590 | 1.6739 | 1.7315 | 2.3185 |
| 2 | 1.0000 | 2.1318 | 2.2034 | 2.5143 | 2.5483 | 2.0000 | 1.3141 | 1.2520 | 1.2052 | 1.6231 | 1.6597 | 2.1951 | |
 |
2.03 | 1.0000 | 2.1181 | 2.1887 | 2.4952 | 2.5287 | 1.9879 | 1.3085 | 1.2474 | 1.2014 | 1.6194 | 1.6545 | 2.1863 |
| 3 | 1.0000 | 1.7571 | 1.8054 | 2.0091 | 2.0308 | 1.6667 | 1.1664 | 1.1302 | 1.1035 | 1.5184 | 1.5245 | 1.9608 | |
 |
3.23 | 1.0000 | 1.6879 | 1.7324 | 1.9191 | 1.9388 | 1.6042 | 1.1404 | 1.1088 | 1.0857 | 1.4983 | 1.5006 | 1.9187 |
 |
3.33 | 1.0000 | 1.6601 | 1.7031 | 1.8832 | 1.9023 | 1.5791 | 1.1302 | 1.1004 | 1.0787 | 1.4903 | 1.4911 | 1.9020 |
| 5 | 1.0000 | 1.3002 | 1.3264 | 1.4332 | 1.4442 | 1.2500 | 1.0088 | 1.0018 | 0.9978 | 1.3829 | 1.3729 | 1.6901 | |
 |
7 | 1.0000 | 1.0318 | 1.0483 | 1.1139 | 1.1205 | 1.0000 | 0.9419 | 0.9500 | 0.9571 | 1.3005 | 1.2910 | 1.5385 |
 |
7.31 | 1.0000 | 1.0001 | 1.0155 | 1.0770 | 1.0832 | 0.9701 | 0.9362 | 0.9458 | 0.9541 | 1.2907 | 1.2816 | 1.5208 |
 |
7.46 | 1.0000 | 0.9851 | 1.0001 | 1.0596 | 1.0656 | 0.9560 | 0.9337 | 0.9440 | 0.9528 | 1.2860 | 1.2771 | 1.5126 |
 |
8.02 | 1.0000 | 0.9334 | 0.9468 | 1.0000 | 1.0054 | 0.9073 | 0.9262 | 0.9389 | 0.9493 | 1.2700 | 1.2620 | 1.4841 |
 |
8.07 | 1.0000 | 0.9288 | 0.9421 | 0.9947 | 1.0000 | 0.9029 | 0.9256 | 0.9385 | 0.9490 | 1.2686 | 1.2606 | 1.4816 |
| 10 | 1.0000 | 0.7879 | 0.7975 | 0.8349 | 0.8386 | 0.7692 | 0.9160 | 0.9338 | 0.9473 | 1.2250 | 1.2199 | 1.4050 | |
| 13 | 1.0000 | 0.6373 | 0.6435 | 0.6677 | 0.6700 | 0.6250 | 0.9269 | 0.9458 | 0.9591 | 1.1788 | 1.1766 | 1.3245 | |
| 15 | 1.0000 | 0.5652 | 0.5701 | 0.5890 | 0.5909 | 0.5556 | 0.9407 | 0.9576 | 0.9690 | 1.1571 | 1.1558 | 1.2865 | |
| 20 | 1.0000 | 0.4407 | 0.4437 | 0.4550 | 0.4561 | 0.4348 | 0.9722 | 0.9818 | 0.9877 | 1.1201 | 1.1199 | 1.2217 | |
| 30 | 1.0000 | 0.3059 | 0.3073 | 0.3127 | 0.3132 | 0.3030 | 0.9967 | 0.9981 | 0.9989 | 1.0814 | 1.0814 | 1.1526 | |
| 50 | 1.0000 | 0.1898 | 0.1903 | 0.1924 | 0.1926 | 0.1887 | 1.0000 | 1.0000 | 1.0000 | 1.0494 | 1.0494 | 1.0940 | |
| 100 | 1.0000 | 0.0974 | 0.0975 | 0.0981 | 0.0981 | 0.0971 | 1.0000 | 1.0000 | 1.0000 | 1.0145 | 1.0145 | 1.0480 | |
First, we note that we have two classes of estimators, namely, the traditional rank‐based PTEs and Stein‐type estimators and the penalty estimators. The restricted R‐estimators play an important role due to the fact that LASSO belongs to the class of restricted estimators. We have the following conclusion from our study.
- Since the inception of the RRE by Hoerl and Kennard (1970), there have been articles comparing RRE with PTE and Stein‐type estimators. We have now definitive conclusion that the RRE dominates the RE, PTRE, and Stein‐type estimators uniformly. See Tables 10.1 and 10.2 and graphs thereof in Figure 10.1. The RRRE ridge estimator dominates the modified RLASSO estimator uniformly for
, while they are 
‐risk equivalent at 
. The RRRE ridge estimator does not select variables, but the modified RLASSO estimator does. - The restricted R‐ and modified RLASSO estimators are competitive, although the modified RLASSO estimator lags behind the restricted R‐estimator, uniformly. Both estimators outperform the URE, PTRE, SRE, and PRSRE in a subinterval of
(see Tables 10.1 and 10.2).
- The lower bound of
risk of HTE and the modified RLASSO estimator is the same and independent of the threshold parameter. But the upper bound of 
risk is dependent on the threshold parameter. - Maximum of RWRE occurs at
, which indicates that the RE underperforms all estimators for any value of 
. Clearly, RRE outperforms all estimators for any 
at 
. However, as 
deviates from 0, the rank‐based PTE and the Stein‐type estimator outperform URE, RRE, and the modified RLASSO estimator (see Tables 10.1 and 10.2). - If
is fixed and 
increases, the RWRE of all estimators increases (see Tables 10.7 and 10.8). - If
is fixed and 
increases, the RWRE of all estimators decreases. Then, for 
small and 
large, the modified RLASSO, PTRE, SRE, and PRSRE are competitive (see Table 10.9 and 10.10). - The PRSRE always outperforms the SRE (see Tables 10.1–10.10).
Table 10.2 RWRE for the estimators for 
and 
.
| RRE | PTRE | ||||||||||||
 |
 |
||||||||||||
 |
RE | 0.1 | 0.2 | 0.8 | 0.9 | MRLASSO |  |
0.2 |  |
SRE | PRSRE | RRRE | |
| 0 | 1.0000 | 2.9890 | 3.0586 | 3.3459 | 3.3759 | 2.8571 | 1.9458 | 1.7926 | 1.6694 | 2.2222 | 2.4326 | 2.8571 | |
| 0.1 | 1.0000 | 2.9450 | 3.0125 | 3.2909 | 3.3199 | 2.8169 | 1.9146 | 1.7658 | 1.6464 | 2.2017 | 2.3958 | 2.8172 | |
| 0.5 | 1.0000 | 2.7811 | 2.8413 | 3.0876 | 3.1132 | 2.6667 | 1.8009 | 1.6683 | 1.5627 | 2.1255 | 2.2661 | 2.6733 | |
| 1 | 1.0000 | 2.6003 | 2.6528 | 2.8664 | 2.8883 | 2.5000 | 1.6797 | 1.5648 | 1.4739 | 2.0423 | 2.1352 | 2.5225 | |
| 2 | 1.0000 | 2.3011 | 2.3421 | 2.5070 | 2.5238 | 2.2222 | 1.4910 | 1.4041 | 1.3365 | 1.9065 | 1.9434 | 2.2901 | |
 |
2.14 | 1.0000 | 2.2642 | 2.3039 | 2.4633 | 2.4795 | 2.1878 | 1.4688 | 1.3854 | 1.3205 | 1.8899 | 1.9217 | 2.2628 |
 |
2.67 | 1.0000 | 2.1354 | 2.1707 | 2.3116 | 2.3259 | 2.0673 | 1.3937 | 1.3218 | 1.2663 | 1.8323 | 1.8488 | 2.1697 |
| 3 | 1.0000 | 2.0637 | 2.0966 | 2.2278 | 2.2410 | 2.0000 | 1.3535 | 1.2878 | 1.2375 | 1.8005 | 1.8100 | 2.1192 | |
 |
4.19 | 1.0000 | 1.8378 | 1.8639 | 1.9669 | 1.9772 | 1.7872 | 1.2352 | 1.1885 | 1.1534 | 1.7014 | 1.6962 | 1.9667 |
 |
4.31 | 1.0000 | 1.8173 | 1.8427 | 1.9433 | 1.9534 | 1.7677 | 1.2251 | 1.1801 | 1.1463 | 1.6925 | 1.6864 | 1.9533 |
| 5 | 1.0000 | 1.7106 | 1.7332 | 1.8219 | 1.8307 | 1.6667 | 1.1750 | 1.1385 | 1.1114 | 1.6464 | 1.6370 | 1.8848 | |
| 7 | 1.0000 | 1.4607 | 1.4771 | 1.5411 | 1.5474 | 1.4286 | 1.0735 | 1.0554 | 1.0425 | 1.5403 | 1.5289 | 1.7316 | |
| 10 | 1.0000 | 1.1982 | 1.2092 | 1.2517 | 1.2559 | 1.1765 | 0.9982 | 0.9961 | 0.9952 | 1.4319 | 1.4243 | 1.5808 | |
| 15 | 1.0000 | 2.0533 | 2.0685 | 2.0958 | 2.0966 | 1.8182 | 1.0934 | 1.0694 | 1.0529 | 2.1262 | 2.1157 | 2.3105 | |
 |
13 | 1.0000 | 1.0156 | 1.0236 | 1.0539 | 1.0568 | 1.0000 | 0.9711 | 0.9767 | 0.9811 | 1.3588 | 1.3547 | 1.4815 |
 |
13.31 | 1.0000 | 1.0000 | 1.0077 | 1.0370 | 1.0399 | 0.9848 | 0.9699 | 0.9760 | 0.9807 | 1.3526 | 1.3488 | 1.4732 |
 |
13.46 | 1.0000 | 0.9925 | 1.0000 | 1.0289 | 1.0318 | 0.9775 | 0.9694 | 0.9757 | 0.9805 | 1.3496 | 1.3459 | 1.4692 |
 |
14.02 | 1.0000 | 0.9655 | 0.9727 | 1.0000 | 1.0027 | 0.9514 | 0.9679 | 0.9749 | 0.9801 | 1.3391 | 1.3358 | 1.4550 |
 |
14.07 | 1.0000 | 0.9631 | 0.9702 | 0.9974 | 1.0000 | 0.9490 | 0.9678 | 0.9748 | 0.9801 | 1.3381 | 1.3349 | 1.4537 |
| 15 | 1.0000 | 0.9220 | 0.9285 | 0.9534 | 0.9558 | 0.9091 | 0.9667 | 0.9745 | 0.9802 | 1.3221 | 1.3195 | 1.4322 | |
| 20 | 1.0000 | 0.7493 | 0.7536 | 0.7699 | 0.7715 | 0.7407 | 0.9743 | 0.9820 | 0.9870 | 1.2560 | 1.2553 | 1.3442 | |
| 30 | 1.0000 | 0.5451 | 0.5473 | 0.5559 | 0.5567 | 0.5405 | 0.9940 | 0.9964 | 0.9977 | 1.1809 | 1.1808 | 1.2446 | |
| 50 | 1.0000 | 0.3528 | 0.3537 | 0.3573 | 0.3576 | 0.3509 | 0.9999 | 1.0000 | 1.0000 | 1.1136 | 1.1136 | 1.1549 | |
| 100 | 1.0000 | 0.1875 | 0.1877 | 0.1887 | 0.1888 | 0.1869 | 1.0000 | 1.0000 | 1.0000 | 1.0337 | 1.0337 | 1.0808 | |
Finally, we present the RWRE formula from which we prepared our tables and figures, for a quick summary.
Now, we describe Table 10.1. This table presents RWRE of the seven estimators for 
, 
, and 
, 
against 
‐values using a sample of size 
, the 
matrix is produced. Using the model given by Eq. (10.62) for chosen values, 
and 
. Therefore, the RWRE value of RLSE has four entries – two for low correlation and two for high correlation. Some 
‐values are given as 
and 
for chosen 
‐values. Now, one may use the table for the performance characteristics of each estimator compared to any other.
Tables 10.2–10.3 give the RWRE values of estimators for 
, and 7 for 
, and 30.
Table 10.3 RWRE of the R‐estimators for 
and different 
‐values for varying 
.
| Estimators |  |
 |
 |
 |
 |
 |
 |
 |
 |
 |
|||||||
| RE | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 |
RRE ( ) |
5.6807 | 3.7105 | 2.1589 | 1.4946 | 3.6213 | 2.7059 | 1.7755 | 1.3003 |
RRE ( ) |
6.1242 | 3.9296 | 2.2357 | 1.5291 | 3.7959 | 2.8204 | 1.8271 | 1.3262 |
RRE ( ) |
9.4863 | 5.0497 | 2.5478 | 1.6721 | 4.8660 | 3.3549 | 2.0304 | 1.4325 |
RRE ( ) |
10.1255 | 5.1921 | 2.5793 | 1.6884 | 5.0292 | 3.4171 | 2.0504 | 1.4445 |
| RLASSO | 5.0000 | 3.3333 | 2.0000 | 1.4286 | 3.3333 | 2.5000 | 1.6667 | 1.2500 |
PTRE ( ) |
2.3441 | 1.9787 | 1.5122 | 1.2292 | 1.7548 | 1.5541 | 1.2714 | 1.0873 |
PTRE ( ) |
2.0655 | 1.7965 | 1.4292 | 1.1928 | 1.6044 | 1.4499 | 1.2228 | 1.0698 |
PTRE ( ) |
1.8615 | 1.6565 | 1.3616 | 1.1626 | 1.4925 | 1.3703 | 1.1846 | 1.0564 |
| SRE | 2.5000 | 2.0000 | 1.4286 | 1.1111 | 2.1364 | 1.7725 | 1.3293 | 1.0781 |
| PRSRE | 3.0354 | 2.3149 | 1.5625 | 1.1625 | 2.3107 | 1.8843 | 1.3825 | 1.1026 |
| RRRE | 5.0000 | 3.3333 | 2.0000 | 1.4286 | 3.4615 | 2.5806 | 1.7143 | 1.2903 |
 |
 |
|||||||
| RE | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 |
RRE ( ) |
1.4787 | 1.2993 | 1.0381 | 0.8554 | 0.8501 | 0.7876 | 0.6834 | 0.5991 |
RRE ( ) |
1.5069 | 1.3251 | 1.0555 | 0.8665 | 0.8594 | 0.7970 | 0.6909 | 0.6046 |
RRE ( ) |
1.6513 | 1.4324 | 1.1204 | 0.9107 | 0.9045 | 0.8346 | 0.7181 | 0.6257 |
RRE ( ) |
1.6698 | 1.4437 | 1.1264 | 0.9155 | 0.9100 | 0.8384 | 0.7206 | 0.6280 |
| RLASSO | 1.4286 | 1.2500 | 1.0000 | 0.8333 | 0.8333 | 0.7692 | 0.6667 | 0.5882 |
PTRE ( ) |
1.0515 | 1.0088 | 0.9465 | 0.9169 | 0.9208 | 0.9160 | 0.9176 | 0.9369 |
PTRE ( ) |
1.0357 | 1.0018 | 0.9530 | 0.9323 | 0.9366 | 0.9338 | 0.9374 | 0.9545 |
PTRE ( ) |
1.0250 | 0.9978 | 0.9591 | 0.9447 | 0.9488 | 0.9473 | 0.9517 | 0.9665 |
| SRE | 1.5516 | 1.3829 | 1.1546 | 1.0263 | 1.3238 | 1.2250 | 1.0865 | 1.0117 |
| PRSRE | 1.5374 | 1.3729 | 1.1505 | 1.0268 | 1.3165 | 1.2199 | 1.0843 | 1.0114 |
| RRRE | 1.9697 | 1.6901 | 1.3333 | 1.1268 | 1.5517 | 1.4050 | 1.2000 | 1.0744 |
 |
 |
|||||||
| RE | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 |
RRE ( ) |
0.4595 | 0.4406 | 0.4060 | 0.3747 | 0.1619 | 0.1595 | 0.1547 | 0.1499 |
RRE ( ) |
0.4622 | 0.4435 | 0.4086 | 0.3768 | 0.1622 | 0.1599 | 0.1551 | 0.1503 |
RRE ( ) |
0.4749 | 0.4549 | 0.4180 | 0.3849 | 0.1638 | 0.1613 | 0.1564 | 0.1516 |
RRE ( ) |
0.4764 | 0.4561 | 0.4188 | 0.3858 | 0.1640 | 0.1615 | 0.1566 | 0.1517 |
| RLASSO | 0.4545 | 0.4348 | 0.4000 | 0.3704 | 0.1613 | 0.1587 | 0.1538 | 0.1493 |
PTRE ( ) |
0.9673 | 0.9722 | 0.9826 | 0.9922 | 1.0000 | 1.0000 | 1.0000 | 1.0000 |
PTRE ( ) |
0.9783 | 0.9818 | 0.9890 | 0.9954 | 1.0000 | 1.0000 | 1.0000 | 1.0000 |
PTRE ( ) |
0.9850 | 0.9877 | 0.9928 | 0.9971 | 1.0000 | 1.0000 | 1.0000 | 1.0000 |
| SRE | 1.1732 | 1.1201 | 1.0445 | 1.0053 | 1.0595 | 1.0412 | 1.0150 | 1.0017 |
| PRSRE | 1.1728 | 1.1199 | 1.0444 | 1.0053 | 1.0595 | 1.0412 | 1.0150 | 1.0017 |
| RRRE | 1.2963 | 1.2217 | 1.1111 | 1.0407 | 1.1039 | 1.0789 | 1.0400 | 1.0145 |
Table 10.4 gives the RWRE values of estimators for 
and 
, and 55, and also for 
and 
, and 53. Also, Table 10.5 presents the RWRE values of estimators for 
and 
, and 57 as well as 
and 
, and 53 to see the effect of 
variation on RWRE (Figures 10.2 and 10.3).
Table 10.4 RWRE of the R‐estimators for 
and different 
‐values for varying 
.
| Estimators |  |
 |
 |
 |
 |
 |
 |
 |
 |
 |
|||||||
| RE | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 |
RRE ( ) |
9.1045 | 5.9788 | 3.4558 | 2.3953 | 5.6626 | 4.2736 | 2.8084 | 2.0654 |
RRE ( ) |
9.8493 | 6.3606 | 3.6039 | 2.4653 | 5.9410 | 4.4650 | 2.9055 | 2.1172 |
RRE ( ) |
15.1589 | 8.0862 | 4.0566 | 2.6637 | 7.5344 | 5.2523 | 3.1928 | 2.2620 |
RRE ( ) |
16.1151 | 8.2786 | 4.0949 | 2.6780 | 7.7633 | 5.3329 | 3.2165 | 2.2723 |
| RLASSO | 7.5000 | 5.0000 | 3.0000 | 2.1429 | 5.0000 | 3.7500 | 2.5000 | 1.8750 |
PTRE ( ) |
2.8417 | 2.4667 | 1.9533 | 1.6188 | 2.1718 | 1.9542 | 1.6304 | 1.4021 |
PTRE ( ) |
2.4362 | 2.1739 | 1.7905 | 1.5242 | 1.9277 | 1.7680 | 1.5192 | 1.3354 |
PTRE ( ) |
2.1488 | 1.9568 | 1.6617 | 1.4462 | 1.7505 | 1.6288 | 1.4325 | 1.2820 |
| SRE | 3.7500 | 3.0000 | 2.1429 | 1.6667 | 3.1296 | 2.5964 | 1.9385 | 1.5495 |
| PRSRE | 4.6560 | 3.5445 | 2.3971 | 1.8084 | 3.4352 | 2.7966 | 2.0402 | 1.6081 |
| RRRE | 7.5000 | 5.0000 | 3.0000 | 2.1429 | 5.1220 | 3.8235 | 2.5385 | 1.9014 |
 |
 |
|||||||
| RE | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 |
RRE ( ) |
2.2555 | 1.9970 | 1.6056 | 1.3318 | 1.2875 | 1.1989 | 1.0458 | 0.9223 |
RRE ( ) |
2.2983 | 2.0378 | 1.6369 | 1.3531 | 1.3013 | 1.2134 | 1.0590 | 0.9325 |
RRE ( ) |
2.5034 | 2.1876 | 1.7244 | 1.4109 | 1.3646 | 1.2651 | 1.0950 | 0.9596 |
RRE ( ) |
2.5282 | 2.2015 | 1.7313 | 1.4149 | 1.3720 | 1.2697 | 1.0977 | 0.9614 |
| RLASSO | 2.1429 | 1.8750 | 1.5000 | 1.2500 | 1.2500 | 1.1538 | 1.0000 | 0.8824 |
PTRE ( ) |
1.2477 | 1.1936 | 1.1034 | 1.0338 | 0.9976 | 0.9828 | 0.9598 | 0.9458 |
PTRE ( ) |
1.1936 | 1.1510 | 1.0793 | 1.0235 | 0.9948 | 0.9836 | 0.9665 | 0.9568 |
PTRE ( ) |
1.1542 | 1.1200 | 1.0619 | 1.0166 | 0.9936 | 0.9850 | 0.9721 | 0.9653 |
| SRE | 2.0987 | 1.8655 | 1.5332 | 1.3106 | 1.6727 | 1.5403 | 1.3382 | 1.1948 |
| PRSRE | 2.0783 | 1.8494 | 1.5227 | 1.3038 | 1.6589 | 1.5294 | 1.3315 | 1.1909 |
| RRRE | 2.6733 | 2.2973 | 1.8000 | 1.4885 | 1.9602 | 1.7742 | 1.5000 | 1.3107 |
 |
 |
|||||||
| RE | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 |
RRE ( ) |
0.6928 | 0.6663 | 0.6162 | 0.5711 | 0.2433 | 0.2400 | 0.2331 | 0.2264 |
RRE ( ) |
0.6968 | 0.6708 | 0.6208 | 0.5750 | 0.2438 | 0.2405 | 0.2338 | 0.2270 |
RRE ( ) |
0.7146 | 0.6863 | 0.6329 | 0.5852 | 0.2459 | 0.2425 | 0.2355 | 0.2285 |
RRE ( ) |
0.7166 | 0.6876 | 0.6339 | 0.5859 | 0.2462 | 0.2427 | 0.2356 | 0.2287 |
| RLASSO | 0.6818 | 0.6522 | 0.6000 | 0.5556 | 0.2419 | 0.2381 | 0.2308 | 0.2239 |
PTRE ( ) |
0.9660 | 0.9675 | 0.9720 | 0.9779 | 1.0000 | 1.0000 | 1.0000 | 1.0000 |
PTRE ( ) |
0.9761 | 0.9774 | 0.9810 | 0.9854 | 1.0000 | 1.0000 | 1.0000 | 1.0000 |
PTRE ( ) |
0.9828 | 0.9839 | 0.9866 | 0.9900 | 1.0000 | 1.0000 | 1.0000 | 1.0000 |
| SRE | 1.3731 | 1.3034 | 1.1917 | 1.1091 | 1.1318 | 1.1082 | 1.0689 | 1.0389 |
| PRSRE | 1.3720 | 1.3026 | 1.1912 | 1.1089 | 1.1318 | 1.1082 | 1.0689 | 1.0389 |
| RRRE | 1.5184 | 1.4286 | 1.2857 | 1.1798 | 1.1825 | 1.1538 | 1.1053 | 1.0669 |
Table 10.5 RWRE of the R‐estimators for 
and different 
‐values for varying 
.
| Estimators |  |
 |
 |
 |
 |
 |
 |
 |
 |
 |
|||||||
| RE | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 |
RRE ( ) |
12.9872 | 8.5010 | 4.9167 | 3.4047 | 7.8677 | 5.9631 | 3.9460 | 2.9092 |
RRE ( ) |
14.0913 | 9.0423 | 5.1335 | 3.5107 | 8.2590 | 6.2241 | 4.0844 | 2.9863 |
RRE ( ) |
21.5624 | 11.4369 | 5.7360 | 3.7651 | 10.3661 | 7.2732 | 4.4569 | 3.1684 |
RRE ( ) |
22.8744 | 11.6969 | 5.7922 | 3.7803 | 10.6590 | 7.3774 | 4.4908 | 3.1791 |
| RLASSO | 10.0000 | 6.6667 | 4.0000 | 2.8571 | 6.6667 | 5.0000 | 3.3333 | 2.5000 |
PTRE ( ) |
3.2041 | 2.8361 | 2.3073 | 1.9458 | 2.4964 | 2.2738 | 1.9310 | 1.6797 |
PTRE ( ) |
2.6977 | 2.4493 | 2.0693 | 1.7926 | 2.1721 | 2.0143 | 1.7602 | 1.5648 |
PTRE ( ) |
2.3469 | 2.1698 | 1.8862 | 1.6694 | 1.9413 | 1.8244 | 1.6295 | 1.4739 |
| SRE | 5.0000 | 4.0000 | 2.8571 | 2.2222 | 4.1268 | 3.4258 | 2.5581 | 2.0423 |
| PRSRE | 6.2792 | 4.7722 | 3.2234 | 2.4326 | 4.5790 | 3.7253 | 2.7140 | 2.1352 |
| RRRE | 10.0000 | 6.6667 | 4.0000 | 2.8571 | 6.7857 | 5.0704 | 3.3684 | 2.5225 |
 |
 |
|||||||
| RE | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 |
RRE ( ) |
3.0563 | 2.7190 | 2.2051 | 1.8390 | 1.7324 | 1.6186 | 1.4214 | 1.2598 |
RRE ( ) |
3.1135 | 2.7720 | 2.2477 | 1.8695 | 1.7507 | 1.6372 | 1.4390 | 1.2740 |
RRE ( ) |
3.3724 | 2.9625 | 2.3561 | 1.9393 | 1.8297 | 1.7019 | 1.4827 | 1.3060 |
RRE ( ) |
3.4028 | 2.9797 | 2.3656 | 1.9433 | 1.8386 | 1.7076 | 1.4864 | 1.3079 |
| RLASSO | 2.8571 | 2.5000 | 2.0000 | 1.6667 | 1.6667 | 1.5385 | 1.3333 | 1.1765 |
PTRE ( ) |
1.4223 | 1.3625 | 1.2595 | 1.1750 | 1.0788 | 1.0592 | 1.0254 | 0.9982 |
PTRE ( ) |
1.3324 | 1.2864 | 1.2058 | 1.1385 | 1.0580 | 1.0429 | 1.0169 | 0.9961 |
PTRE ( ) |
1.2671 | 1.2306 | 1.1660 | 1.1114 | 1.0437 | 1.0319 | 1.0114 | 0.9952 |
| SRE | 2.6519 | 2.3593 | 1.9363 | 1.6464 | 2.0283 | 1.8677 | 1.6170 | 1.4319 |
| PRSRE | 2.6304 | 2.3416 | 1.9237 | 1.6370 | 2.0082 | 1.8513 | 1.6059 | 1.4243 |
| RRRE | 3.3824 | 2.9139 | 2.2857 | 1.8848 | 2.3729 | 2.1514 | 1.8182 | 1.5808 |
 |
 |
|||||||
| RE | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 |
RRE ( ) |
0.9283 | 0.8946 | 0.8309 | 0.7729 | 0.3250 | 0.3207 | 0.3122 | 0.3036 |
RRE ( ) |
0.9335 | 0.9002 | 0.8369 | 0.7782 | 0.3256 | 0.3215 | 0.3130 | 0.3044 |
RRE ( ) |
0.9555 | 0.9195 | 0.8515 | 0.7901 | 0.3282 | 0.3239 | 0.3150 | 0.3062 |
RRE ( ) |
0.9580 | 0.9211 | 0.8527 | 0.7908 | 0.3285 | 0.3241 | 0.3152 | 0.3063 |
| RLASSO | 0.9091 | 0.8696 | 0.8000 | 0.7407 | 0.3226 | 0.3175 | 0.3077 | 0.2985 |
PTRE ( ) |
0.9747 | 0.9737 | 0.9731 | 0.9743 | 1.0000 | 1.0000 | 1.0000 | 1.0000 |
PTRE ( ) |
0.9813 | 0.9808 | 0.9808 | 0.9820 | 1.0000 | 1.0000 | 1.0000 | 1.0000 |
PTRE ( ) |
0.9860 | 0.9857 | 0.9859 | 0.9870 | 1.0000 | 1.0000 | 1.0000 | 1.0000 |
| SRE | 1.5796 | 1.4978 | 1.3623 | 1.2560 | 1.2078 | 1.1811 | 1.1344 | 1.0957 |
| PRSRE | 1.5775 | 1.4961 | 1.3612 | 1.2553 | 1.2078 | 1.1811 | 1.1344 | 1.0957 |
| RRRE | 1.7431 | 1.6408 | 1.4737 | 1.3442 | 1.2621 | 1.2310 | 1.1765 | 1.1309 |
Table 10.6 RWRE of the R‐estimators for 
and different 
‐values for varying 
.
| Estimators |  |
 |
 |
 |
 |
 |
 |
 |
 |
 |
|||||||
| RE | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 |
RRE ( ) |
22.3753 | 14.6442 | 8.4349 | 5.8203 | 12.8002 | 9.8339 | 6.5821 | 4.8740 |
RRE ( ) |
24.3855 | 15.6276 | 8.7997 | 6.0133 | 13.4302 | 10.2680 | 6.8022 | 5.0086 |
RRE ( ) |
37.2461 | 19.5764 | 9.7789 | 6.3861 | 16.5880 | 11.8373 | 7.3729 | 5.2646 |
RRE ( ) |
39.3232 | 20.0631 | 9.8674 | 6.4129 | 16.9876 | 12.0125 | 7.4229 | 5.2828 |
| RLASSO | 15.0000 | 10.0000 | 6.0000 | 4.2857 | 10.0000 | 7.5000 | 5.0000 | 3.7500 |
PTRE ( ) |
3.7076 | 3.3671 | 2.8451 | 2.4637 | 2.9782 | 2.7592 | 2.4060 | 2.1337 |
PTRE ( ) |
3.0508 | 2.8314 | 2.4758 | 2.2002 | 2.5244 | 2.3765 | 2.1279 | 1.9271 |
PTRE ( ) |
2.6089 | 2.4579 | 2.2032 | 1.9968 | 2.2107 | 2.1051 | 1.9219 | 1.7688 |
| SRE | 7.5000 | 6.0000 | 4.2857 | 3.3333 | 6.1243 | 5.0891 | 3.8037 | 3.0371 |
| PRSRE | 9.5356 | 7.2308 | 4.8740 | 3.6755 | 6.8992 | 5.6069 | 4.0788 | 3.2054 |
| RRRE | 15.0000 | 10.0000 | 6.0000 | 4.2857 | 10.1163 | 7.5676 | 5.0323 | 3.7696 |
 |
 |
|||||||
| RE | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 |
RRE ( ) |
4.7273 | 4.2535 | 3.5048 | 2.9537 | 2.6439 | 2.4888 | 2.2123 | 1.9792 |
RRE ( ) |
4.8104 | 4.3328 | 3.5663 | 3.0026 | 2.6697 | 2.5158 | 2.2366 | 2.0011 |
RRE ( ) |
5.1633 | 4.5899 | 3.7172 | 3.0928 | 2.7751 | 2.6004 | 2.2951 | 2.0407 |
RRE ( ) |
5.2015 | 4.6159 | 3.7298 | 3.0991 | 2.7861 | 2.6087 | 2.2999 | 2.0435 |
| RLASSO | 4.2857 | 3.7500 | 3.0000 | 2.5000 | 2.5000 | 2.3077 | 2.0000 | 1.7647 |
PTRE ( ) |
1.7190 | 1.6538 | 1.5384 | 1.4395 | 1.2353 | 1.2113 | 1.1675 | 1.1287 |
PTRE ( ) |
1.5642 | 1.5158 | 1.4286 | 1.3524 | 1.1806 | 1.1622 | 1.1285 | 1.0984 |
PTRE ( ) |
1.4532 | 1.4159 | 1.3478 | 1.2873 | 1.1418 | 1.1273 | 1.1007 | 1.0768 |
| SRE | 3.7621 | 3.3545 | 2.7588 | 2.3444 | 2.7433 | 2.5307 | 2.1934 | 1.9380 |
| PRSRE | 3.7497 | 3.3433 | 2.7492 | 2.3362 | 2.7120 | 2.5044 | 2.1742 | 1.9237 |
| RRRE | 4.8058 | 4.1558 | 3.2727 | 2.7010 | 3.2022 | 2.9134 | 2.4706 | 2.1475 |
 |
 |
|||||||
| RE | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 |
RRE ( ) |
1.4053 | 1.3603 | 1.2733 | 1.1925 | 0.4890 | 0.4834 | 0.4720 | 0.4604 |
RRE ( ) |
1.4126 | 1.3683 | 1.2813 | 1.2004 | 0.4899 | 0.4845 | 0.4731 | 0.4616 |
RRE ( ) |
1.4416 | 1.3930 | 1.3003 | 1.2145 | 0.4933 | 0.4875 | 0.4756 | 0.4637 |
RRE ( ) |
1.4445 | 1.3953 | 1.3018 | 1.2155 | 0.4937 | 0.4878 | 0.4759 | 0.4638 |
| RLASSO | 1.3636 | 1.3043 | 1.2000 | 1.1111 | 0.4839 | 0.4762 | 0.4615 | 0.4478 |
PTRE ( ) |
1.0081 | 1.0040 | 0.9969 | 0.9912 | 0.9999 | 0.9999 | 0.9999 | 1.0000 |
PTRE ( ) |
1.0047 | 1.0018 | 0.9968 | 0.9928 | 1.0000 | 1.0000 | 1.0000 | 1.0000 |
PTRE ( ) |
1.0027 | 1.0006 | 0.9970 | 0.9942 | 1.0000 | 1.0000 | 1.0000 | 1.0000 |
| SRE | 1.9969 | 1.8948 | 1.7215 | 1.5804 | 1.3630 | 1.3320 | 1.2759 | 1.2268 |
| PRSRE | 1.9921 | 1.8907 | 1.7185 | 1.5782 | 1.3630 | 1.3320 | 1.2759 | 1.2268 |
| RRRE | 2.1951 | 2.0705 | 1.8621 | 1.6951 | 1.4224 | 1.3876 | 1.3247 | 1.2698 |
Figure 10.2 RWRE for the modified RLASSO (MRLASSO), ridge, restricted, preliminary test and the Stein‐type and its positive rule estimators.
Figure 10.3 RWRE of R‐estimates of a function of 
for 
, 
, and different 
.
Table 10.7 RWRE values of estimators for 
and different values of 
and 
.
| RRE | PTRE | |||||||||
 |
 |
|||||||||
 |
RE | 0.8 | 0.9 | MRLASSO | 0.15 | 0.2 | 0.25 | SRE | PRSRE | RRRE |
 |
||||||||||
| 7 | 1.0000 | 4.0468 | 4.1621 | 3.3333 | 2.6044 | 1.9787 | 1.6565 | 2.0000 | 2.3149 | 3.3333 |
| 17 | 1.0000 | 10.2978 | 10.5396 | 6.6667 | 4.3870 | 2.8361 | 2.1698 | 4.0000 | 4.7722 | 6.6667 |
| 27 | 1.0000 | 18.3009 | 18.7144 | 10.0000 | 5.7507 | 3.3671 | 2.4579 | 6.0000 | 7.2308 | 10.0000 |
| 37 | 1.0000 | 28.9022 | 29.4914 | 13.3333 | 6.8414 | 3.7368 | 2.6477 | 8.0000 | 9.6941 | 13.3333 |
| 57 | 1.0000 | 64.9968 | 66.2169 | 20.0000 | 8.4968 | 4.2281 | 2.8887 | 12.0000 | 14.6307 | 20.0000 |
 |
||||||||||
| 7 | 1.0000 | 3.2295 | 3.3026 | 2.8571 | 2.2117 | 1.7335 | 1.4923 | 1.8733 | 2.0602 | 2.8846 |
| 17 | 1.0000 | 8.0048 | 8.1499 | 5.7143 | 3.7699 | 2.5211 | 1.9784 | 3.6834 | 4.1609 | 5.7377 |
| 27 | 1.0000 | 13.7847 | 14.0169 | 8.5714 | 4.9963 | 3.0321 | 2.2657 | 5.4994 | 6.2907 | 8.5938 |
| 37 | 1.0000 | 20.9166 | 21.2239 | 11.4286 | 5.9990 | 3.3986 | 2.4608 | 7.3167 | 8.4350 | 11.4504 |
| 57 | 1.0000 | 41.5401 | 42.0259 | 17.1429 | 7.5598 | 3.8998 | 2.7153 | 10.9521 | 12.7487 | 17.1642 |
 |
||||||||||
| 7 | 1.0000 | 2.6869 | 2.7373 | 2.5000 | 1.9266 | 1.5541 | 1.3703 | 1.7725 | 1.8843 | 2.5806 |
| 17 | 1.0000 | 6.5476 | 6.6443 | 5.0000 | 3.3014 | 2.2738 | 1.8244 | 3.4258 | 3.7253 | 5.0704 |
| 27 | 1.0000 | 11.0584 | 11.2071 | 7.5000 | 4.4087 | 2.7592 | 2.1051 | 5.0891 | 5.6069 | 7.5676 |
| 37 | 1.0000 | 16.3946 | 16.5829 | 10.0000 | 5.3306 | 3.1162 | 2.3008 | 6.7542 | 7.5071 | 10.0662 |
| 57 | 1.0000 | 30.5579 | 30.8182 | 15.0000 | 6.7954 | 3.6170 | 2.5624 | 10.0860 | 11.3392 | 15.0649 |
 |
||||||||||
| 7 | 1.0000 | 1.1465 | 1.1556 | 1.2500 | 1.0489 | 1.0088 | 0.9978 | 1.3829 | 1.3729 | 1.6901 |
| 17 | 1.0000 | 2.6666 | 2.6824 | 2.5000 | 1.6731 | 1.3625 | 1.2306 | 2.3593 | 2.3416 | 2.9139 |
| 27 | 1.0000 | 4.2854 | 4.3074 | 3.7500 | 2.2360 | 1.6538 | 1.4159 | 3.3545 | 3.3433 | 4.1558 |
| 37 | 1.0000 | 6.0123 | 6.0376 | 5.0000 | 2.7442 | 1.8968 | 1.5652 | 4.3528 | 4.3574 | 5.4019 |
| 57 | 1.0000 | 9.8282 | 9.8547 | 7.5000 | 3.6311 | 2.2844 | 1.7942 | 6.3514 | 6.4081 | 7.8981 |
Table 10.8 RWRE values of estimators for 
and different values of 
and 
.
| RRE | PTRE | |||||||||
 |
 |
|||||||||
 |
RE | 0.8 | 0.9 | MRLASSO | 0.15 | 0.2 | 0.25 | SRE | PRSRE | RRRE |
 |
||||||||||
| 3 | 1.0000 | 2.0072 | 2.0250 | 1.4286 | 1.3333 | 1.2292 | 1.1626 | 1.1111 | 1.1625 | 1.4286 |
| 13 | 1.0000 | 4.1385 | 4.1573 | 2.8571 | 2.4147 | 1.9458 | 1.6694 | 2.2222 | 2.4326 | 2.8571 |
| 23 | 1.0000 | 6.8094 | 6.8430 | 4.2857 | 3.3490 | 2.4637 | 1.9968 | 3.3333 | 3.6755 | 4.2857 |
| 33 | 1.0000 | 10.2779 | 10.3259 | 5.7143 | 4.1679 | 2.8587 | 2.2279 | 4.4444 | 4.9176 | 5.7143 |
| 53 | 1.0000 | 21.7368 | 21.7965 | 8.5714 | 5.5442 | 3.4283 | 2.5371 | 6.6667 | 7.4030 | 8.5714 |
 |
||||||||||
| 3 | 1.0000 | 1.8523 | 1.8674 | 1.3333 | 1.2307 | 1.1485 | 1.1015 | 1.0928 | 1.1282 | 1.3462 |
| 13 | 1.0000 | 3.7826 | 3.7983 | 2.6667 | 2.2203 | 1.8009 | 1.5627 | 2.1255 | 2.2661 | 2.6733 |
| 23 | 1.0000 | 6.1542 | 6.1815 | 4.0000 | 3.0831 | 2.2860 | 1.8742 | 3.1754 | 3.4159 | 4.0057 |
| 33 | 1.0000 | 9.1562 | 9.1942 | 5.3333 | 3.8441 | 2.6619 | 2.0987 | 4.2270 | 4.5705 | 5.3386 |
| 53 | 1.0000 | 18.4860 | 18.5298 | 8.0000 | 5.1338 | 3.2133 | 2.4055 | 6.3313 | 6.8869 | 8.0050 |
 |
||||||||||
| 3 | 1.0000 | 1.7196 | 1.7326 | 1.2500 | 1.1485 | 1.0873 | 1.0564 | 1.0781 | 1.1026 | 1.2903 |
| 13 | 1.0000 | 3.4830 | 3.4963 | 2.5000 | 2.0544 | 1.6797 | 1.4739 | 2.0423 | 2.1352 | 2.5225 |
| 23 | 1.0000 | 5.6140 | 5.6367 | 3.7500 | 2.8534 | 2.1337 | 1.7688 | 3.0371 | 3.2054 | 3.7696 |
| 33 | 1.0000 | 8.2554 | 8.2862 | 5.0000 | 3.5625 | 2.4906 | 1.9856 | 4.0350 | 4.2840 | 5.0185 |
| 53 | 1.0000 | 16.0828 | 16.1162 | 7.5000 | 4.7733 | 3.0225 | 2.2877 | 6.0331 | 6.4531 | 7.5174 |
 |
||||||||||
| 3 | 1.0000 | 1.0930 | 1.0983 | 0.8333 | 0.8688 | 0.9169 | 0.9447 | 1.0263 | 1.0268 | 1.1268 |
| 13 | 1.0000 | 2.1324 | 2.1374 | 1.6667 | 1.3171 | 1.1750 | 1.1114 | 1.6464 | 1.6370 | 1.8848 |
| 23 | 1.0000 | 3.2985 | 3.3063 | 2.5000 | 1.7768 | 1.4395 | 1.2873 | 2.3444 | 2.3362 | 2.7010 |
| 33 | 1.0000 | 4.6206 | 4.6302 | 3.3333 | 2.2057 | 1.6701 | 1.4360 | 3.0520 | 3.0505 | 3.5267 |
| 53 | 1.0000 | 7.8891 | 7.8973 | 5.0000 | 2.9764 | 2.0491 | 1.6701 | 4.4745 | 4.4980 | 5.1863 |
Table 10.9 RWRE values of estimators for 
and different values of 
and 
.
| RRE | PTRE | |||||||||
 |
 |
|||||||||
 |
RE | 0.8 | 0.9 | MRLASSO | 0.15 | 0.2 | 0.25 | SRE | PRSRE | RRRE |
 |
||||||||||
| 7 | 1.0000 | 2.0384 | 2.0631 | 1.4286 | 1.3333 | 1.2292 | 1.1626 | 1.1111 | 1.1625 | 1.4286 |
| 17 | 1.0000 | 1.3348 | 1.3378 | 1.1765 | 1.1428 | 1.1028 | 1.0752 | 1.0526 | 1.0751 | 1.1765 |
| 27 | 1.0000 | 1.2136 | 1.2150 | 1.1111 | 1.0909 | 1.0663 | 1.0489 | 1.0345 | 1.0489 | 1.1111 |
| 37 | 1.0000 | 1.1638 | 1.1642 | 1.0811 | 1.0667 | 1.0489 | 1.0362 | 1.0256 | 1.0362 | 1.0811 |
| 57 | 1.0000 | 1.1188 | 1.1190 | 1.0526 | 1.0435 | 1.0321 | 1.0239 | 1.0169 | 1.0238 | 1.0526 |
 |
||||||||||
| 7 | 1.0000 | 1.8080 | 1.8274 | 1.3333 | 1.2307 | 1.1485 | 1.1015 | 1.0928 | 1.1282 | 1.3462 |
| 17 | 1.0000 | 1.2871 | 1.2898 | 1.1429 | 1.1034 | 1.0691 | 1.0483 | 1.0443 | 1.0602 | 1.1475 |
| 27 | 1.0000 | 1.1879 | 1.1892 | 1.0909 | 1.0667 | 1.0450 | 1.0317 | 1.0291 | 1.0394 | 1.0938 |
| 37 | 1.0000 | 1.1462 | 1.1467 | 1.0667 | 1.0492 | 1.0334 | 1.0236 | 1.0217 | 1.0292 | 1.0687 |
| 57 | 1.0000 | 1.1081 | 1.1083 | 1.0435 | 1.0323 | 1.0220 | 1.0156 | 1.0144 | 1.0193 | 1.0448 |
 |
||||||||||
| 7 | 1.0000 | 1.6244 | 1.6401 | 1.2500 | 1.1485 | 1.0873 | 1.0564 | 1.0781 | 1.1026 | 1.2903 |
| 17 | 1.0000 | 1.2427 | 1.2452 | 1.1111 | 1.0691 | 1.0418 | 1.0274 | 1.0376 | 1.0488 | 1.1268 |
| 27 | 1.0000 | 1.1632 | 1.1645 | 1.0714 | 1.0450 | 1.0275 | 1.0181 | 1.0248 | 1.0320 | 1.0811 |
| 37 | 1.0000 | 1.1292 | 1.1296 | 1.0526 | 1.0334 | 1.0205 | 1.0135 | 1.0185 | 1.0238 | 1.0596 |
| 57 | 1.0000 | 1.0976 | 1.0978 | 1.0345 | 1.0220 | 1.0136 | 1.0090 | 1.0122 | 1.0158 | 1.0390 |
 |
||||||||||
| 7 | 1.0000 | 0.8963 | 0.9011 | 0.8333 | 0.8688 | 0.9169 | 0.9447 | 1.0263 | 1.0268 | 1.1268 |
| 17 | 1.0000 | 0.9738 | 0.9753 | 0.9091 | 0.9298 | 0.9567 | 0.9716 | 1.0130 | 1.0132 | 1.0596 |
| 27 | 1.0000 | 0.9974 | 0.9984 | 0.9375 | 0.9521 | 0.9707 | 0.9809 | 1.0086 | 1.0088 | 1.0390 |
| 37 | 1.0000 | 1.0092 | 1.0096 | 0.9524 | 0.9636 | 0.9778 | 0.9856 | 1.0064 | 1.0066 | 1.0289 |
| 57 | 1.0000 | 1.0204 | 1.0206 | 0.9677 | 0.9754 | 0.9851 | 0.9903 | 1.0043 | 1.0044 | 1.0191 |
Table 10.10 RWRE values of estimators for 
and different values of 
and 
.
| RRE | PTRE | |||||||||
 |
 |
|||||||||
 |
RE | 0.8 | 0.9 | MRLASSO | 0.15 | 0.2 | 0.25 | SRE | PRSRE | RRRE |
 |
||||||||||
| 3 | 1.0000 | 5.0591 | 5.1915 | 3.3333 | 2.6044 | 1.9787 | 1.6565 | 2.0000 | 2.3149 | 3.3333 |
| 13 | 1.0000 | 1.7676 | 1.7705 | 1.5385 | 1.4451 | 1.3286 | 1.2471 | 1.3333 | 1.3967 | 1.5385 |
| 23 | 1.0000 | 1.4489 | 1.4506 | 1.3043 | 1.2584 | 1.1974 | 1.1522 | 1.2000 | 1.2336 | 1.3043 |
| 33 | 1.0000 | 1.3296 | 1.3304 | 1.2121 | 1.1820 | 1.1411 | 1.1100 | 1.1429 | 1.1655 | 1.2121 |
| 53 | 1.0000 | 1.2279 | 1.2281 | 1.1321 | 1.1144 | 1.0898 | 1.0707 | 1.0909 | 1.1046 | 1.1321 |
 |
||||||||||
| 3 | 1.0000 | 4.0373 | 4.1212 | 2.8571 | 2.2117 | 1.7335 | 1.4923 | 1.8733 | 2.0602 | 2.8846 |
| 13 | 1.0000 | 1.6928 | 1.6954 | 1.4815 | 1.3773 | 1.2683 | 1.1975 | 1.3039 | 1.3464 | 1.4851 |
| 23 | 1.0000 | 1.4147 | 1.4164 | 1.2766 | 1.2234 | 1.1642 | 1.1235 | 1.1840 | 1.2071 | 1.2784 |
| 33 | 1.0000 | 1.3078 | 1.3086 | 1.1940 | 1.1587 | 1.1183 | 1.0899 | 1.1319 | 1.1476 | 1.1952 |
| 53 | 1.0000 | 1.2155 | 1.2156 | 1.1215 | 1.1005 | 1.0759 | 1.0582 | 1.0842 | 1.0938 | 1.1222 |
 |
||||||||||
| 3 | 1.0000 | 3.3589 | 3.4169 | 2.5000 | 1.9266 | 1.5541 | 1.3703 | 1.7725 | 1.8843 | 2.5806 |
| 13 | 1.0000 | 1.6240 | 1.6265 | 1.4286 | 1.3166 | 1.2169 | 1.1562 | 1.2786 | 1.3066 | 1.4414 |
| 23 | 1.0000 | 1.3822 | 1.3837 | 1.2500 | 1.1909 | 1.1349 | 1.0990 | 1.1700 | 1.1854 | 1.2565 |
| 33 | 1.0000 | 1.2868 | 1.2876 | 1.1765 | 1.1367 | 1.0979 | 1.0725 | 1.1223 | 1.1329 | 1.1808 |
| 53 | 1.0000 | 1.2033 | 1.2034 | 1.1111 | 1.0871 | 1.0632 | 1.0472 | 1.0783 | 1.0849 | 1.1137 |
 |
||||||||||
| 3 | 1.0000 | 1.4331 | 1.4436 | 1.2500 | 1.0489 | 1.0088 | 0.9978 | 1.3829 | 1.3729 | 1.6901 |
| 13 | 1.0000 | 1.2259 | 1.2272 | 1.1111 | 1.0239 | 1.0044 | 0.9989 | 1.1607 | 1.1572 | 1.2565 |
| 23 | 1.0000 | 1.1671 | 1.1682 | 1.0714 | 1.0158 | 1.0029 | 0.9993 | 1.1017 | 1.0996 | 1.1576 |
| 33 | 1.0000 | 1.1401 | 1.1407 | 1.0526 | 1.0118 | 1.0022 | 0.9994 | 1.0744 | 1.0729 | 1.1137 |
| 53 | 1.0000 | 1.1139 | 1.1141 | 1.0345 | 1.0078 | 1.0015 | 0.9996 | 1.0484 | 1.0474 | 1.0730 |
Problems
- 10.1
Consider model (10.1) and prove that
- 10.2 Prove that
where
- 10.3 For the model (10.1), show that the test of the null‐hypothesis
vs. 
, is given by
where
, 
, and
- 10.4 Show that the ADR of
is given by
where
is the c.d.f. of chi‐square distribution with 
DF and noncentrality parameter 
evaluated at 
. - 10.5 Prove that the
of 
is given by
where
- 10.6 Verify that
of 
is given by
where
is given by (9.70). - 10.7 Prove that the PTRE dominates the RE whenever
Otherwise, RE outperforms the PTRE in the given interval.
- 10.8 Show that modified RLASSO estimator dominates over PTRE whenever
Otherwise, PTRE will dominate the modified RLASSO estimator.
- 10.9 Derive the ADB and ADR of the shrinkage R‐estimators in (10.29), (10.30), and (10.36).
- 10.10 Derive the ADB and ADR of the shrinkage R‐estimators in (10.37), (10.38), and (10.39).
- 10.11 Consider a real data set, where the design matrix elements are moderate to highly correlated, then find the efficiency of the estimators using unweighted risk functions. Find parallel formulas for the efficiency expressions and compare the results with that of the efficiency using the weighted risk function. Are the two results consistent?