4
Seemingly Unrelated Simple Linear Models
In this chapter, we consider an important class of models, namely, the seemingly unrelated simple linear models, belonging to the class of linear hypothesis, useful in the analysis of bioassay data, shelf‐life determination of pharmaceutical products, and profitability analysis of factory products in terms of costs and outputs, among other applications. In this model, as in the analysis of variance (ANOVA) model, 
independent bivariate samples 
are considered such that 
for each pair 
with fixed 
.
The parameters 
and 
are the intercept and slope vectors of the 
‐lines, respectively, and 
is the common known variance. In this model, it is common to test the parallelism hypotheses 
against the alternative hypothesis, 
. Instead, in many applications, one may suspect some of the elements of the 
‐vector may not be significantly different from 
, i.e. 
‐vector may be sparse; in other words, we partition 
and our suspects, 
. Then, the test statistics 
tests the null hypothesis 
vs. 
. Besides this, the main objective of this chapter is to study some penalty estimators and the preliminary test estimator (PTE) and Stein‐type estimator (SE) of 
and 
when one suspects that 
may be 
and compare their properties based on 
‐risk function. For more literature and research on seemingly unrelated linear regression or other models, we refer the readers to Baltagi (1980), Foschi et al. (2003), Kontoghiorghes (2000, 2004), andKontoghiorghes and Clarke (1995), among others.
4.1 Model, Estimation, and Test of Hypothesis
Consider the seemingly unrelated simple linear models
where 
, 
an 
‐tuple of 1s, 
, and 
, 
is the 
dimensional identity matrices so that 
.
4.1.1 LSE of 
and 
It is easy to see from Saleh (2006, Chapter 06) that the least squares estimator (LSE) of 
and 
are
and
respectively, where
4.1.2 Penalty Estimation of 
and 
Following Donoho and Johnstone (1994), Tibshirani (1996) and Saleh et al. (2017), we define the least absolute shrinkage and selection operator (LASSO) estimator of 
as
where 
with 
.
Thus, we write
where 
.
On the other hand, the ridge estimator of 
may be defined as
where the LSE of 
is 
and restricted least squares estimator (RLSE) is 
.
Consequently, the estimator of 
is given by
where 
and 
.
Similarly, the ridge estimator of 
is given by
4.1.3 PTE and Stein‐Type Estimators of 
and 
For the test of 
, where 
, we use the following test statistic:
where 
and the distribution of 
follows a noncentral 
distribution with 
degrees of freedom (DF) and noncentrality parameter 
. Then we can define the PTE, SE, and PRSE (positive‐rule Stein‐type estimator)of 
as
respectively.
For the PTE, SE and PRSE of 
are
4.2 Bias and MSE Expressions of the Estimators
In this section, we present the expressions of bias and mean squared error (MSE) for all the estimators of 
and 
as follows.
Next, we have the following theorem for the 
risk of the estimators.
Under the assumption of Theorem 4.1, we have following 
‐risk expressions for the estimators defined in Sections using the formula
where 
and 
are the weight matrices.
The 
risk of LSE is
when 
and
when 
.
The 
risk of RLSE is
when 
and 
and
when 
and 
.
The 
risk of PTE is
where 
and
when 
, 
, 
and 
.
The 
risk of PRSE is
where 
, and 
.
The LASSO 
risk expression for 
is
The LASSO 
risk expression for 
is
where
The corresponding lower bound of the unweighted risk functions of 
and 
are, respectively,
We will consider the lower bound of 
risk of LASSO to compare with the 
risk of other estimators. Consequently, the lower bound of the weighted 
risk is given by
which is same as the 
risk of the ridge regression estimator (RRE).
The 
risk of RRE is
when 
and 
and
when 
, 
.
4.3 Comparison of Estimators
In this section, we compare various estimators with respect to the LSE, in terms of relative weighted 
‐risk efficiency (RWRE).
4.3.1 Comparison of LSE with RLSE
Recall that the RLSE is given by 
. In this case, the RWRE of RLSE vs. LSE is given by
which is a decreasing function of 
. So, 
.
4.3.2 Comparison of LSE with PTE
The RWRE expression for PTE vs. LSE is given by
where
Then, the PTE outperforms the LSE for
Otherwise, LSE outperforms the PTE in the interval 
.
We may mention that 
is a decreasing function of 
with a maximum at 
, then decreases crossing the 1‐line to a minimum at 
with a value 
, and then increases toward the 1‐line. This means the gains in efficiency of PTE is the highest in the interval given by Eq. (4.24) and loss in efficiency can be noticed outside it.
The 
belongs to the interval
where 
depends on the size 
and given by
The quantity 
is the value 
at which the RWRE value is minimum.
4.3.3 Comparison of LSE with SE and PRSE
We obtain the RWRE as follows:
It is a decreasing function of 
. At 
, its value is 
; and when 
, its value goes to 1. Hence, for 
,
Hence, the gains in efficiency is the highest when 
is small and drops toward 1 when 
is the largest. Also,
So that,
We also provide a graphical representation (Figure 4.1) of RWRE of the estimators.
Figure 4.1 RWRE for the restricted, preliminary test, and Stein‐type and its positive‐rule estimators.
In the next three subsections, we show that the RRE uniformly dominates all other estimators, although it does not select variables.
4.3.4 Comparison of LSE and RLSE with RRE
First, we consider weighted 
‐risk difference of LSE and RRE given by
Hence, RRE outperforms the LSE uniformly. Similarly, for the RLSE and RRE, the weighted 
‐risk difference is given by
Therefore, RRE performs better than RLSE uniformly.
In addition, the RWRE of RRE vs. LSE equals
which is a decreasing function of 
with maximum 
at 
and minimum 1 as 
. So,
4.3.5 Comparison of RRE with PTE, SE, and PRSE
4.3.5.1 Comparison Between 
and 
Here, the weighted 
‐risk difference of 
and 
is given by
Note that the risk of 
is an increasing function of 
crossing the 
‐line to a maximum and then drops monotonically toward the 
‐line as 
. The value of the risk is 
at 
. On the other hand, 
is an increasing function of 
below the 
‐line with a minimum value 0 at 
and as 
, 
. Hence, the risk difference in Eq. (4.30) is nonnegative for 
. Thus, the RRE uniformly performs better than the PTE.
4.3.5.2 Comparison Between 
and 
The weighted 
‐risk difference of 
and 
is given by
Note that the first function is increasing in 
with a value 2 at 
; and as 
, it tends to 
. The second function is also increasing in 
with a value 0 at 
and approaches the value 
as 
. Hence, the risk difference is nonnegative for all 
. Consequently, RRE outperforms SE uniformly.
4.3.5.3 Comparison of 
with 
The risk of 
is
where
and R(
) is
The weighted 
‐risk difference of PR and RRE is given by
where
Consider the 
(
). It is a monotonically increasing function of 
. At 
, its value is 
; and as 
, it tends to 
. For 
, at 
, the value is 
; and as 
, it tends to 
. Hence, the 
‐risk difference in (4.31) is nonnegative and RRE uniformly outperforms PRSE.
Note that the risk difference of 
and 
at 
is
because the expected value in Eq. (4.35) is a decreasing function of DF, and 
. The risk functions of RRE, PT, SE, and PRSE are plotted in Figures 4.2 and 4.3 for 
, 
, respectively. These figures are in support of the given comparisons.
4.3.6 Comparison of LASSO with RRE
Here, the weighted 
‐risk difference is given by
Hence, the RRE outperforms the LASSO uniformly.
Figure 4.2 Weighted 
risk for the ridge, preliminary test, and Stein‐type and its positive‐rule estimators for 
, and 
.
Figure 4.3 Weighted 
risk for the ridge, preliminary test, and Stein‐type and its positive‐rule estimators for 
, and 
.
4.3.7 Comparison of LASSO with LSE and RLSE
First, note that if we have for 
coefficients, 
and also 
coefficients are zero in a sparse solution, then the “ideal” weighted 
‐risk is given by 
. Thereby, we compare all estimators relative to this quantity. Hence, the weighted 
‐risk difference between LSE and LASSO is given by
Hence, if 
, the LASSO performs better than the LSE, while if 
the LSE performs better than the LASSO. Consequently, neither LSE nor the LASSO performs better than the other, uniformly.
Next, we compare the RLSE and LASSO. In this case, the weighted 
‐risk difference is given by
Hence, LASSO and RLSE are 
risk equivalent. And consequently, the LASSO satisfies the oracle properties.
4.3.8 Comparison of LASSO with PTE, SE, and PRSE
We first consider the PTE vs. LASSO. In this case, the weighted 
‐risk difference is given by
Hence, the LASSO outperforms the PTE when 
. But when 
, the LASSO outperforms the PTE for
Otherwise, PTE outperforms the LASSO. Hence, neither LASSO nor PTE outmatches the other uniformly.
Next, we consider SE and PRSE vs. the LASSO. In these two cases, we have weighted 
‐risk differences given by
and
Therefore, the LASSO outperforms the SE as well as the PRSE in the interval 
. Thus, neither SE nor the PRSE outperform the LASSO uniformly.
In Figure 4.4, the comparisons of LASSO with other estimators are shown.
Figure 4.4 RWRE for the LASSO, ridge, restricted, preliminary test, and Stein‐type and its positive‐rule estimators.
4.4 Efficiency in Terms of Unweighted 
Risk
In the previous sections, we have made all comparisons among the estimators in terms of weighted risk functions. In this section, we provide the 
‐risk efficiency of the estimators in terms of the unweighted (weight = 
) risk expressions for both 
and 
.
4.4.1 Efficiency for 
The unweighted relative efficiency of LASSO:
Note that the unweighted risk of LASSO and RLSE is the same. The unweighted relative efficiency of PTE:
The unweighted relative efficiency of SE:
where
The unweighted relative efficiency of PRSE:
The unweighted relative efficiency of RRE:
4.4.2 Efficiency for 
The unweighted relative efficiency of LASSO:
Note that the unweighted risk of LASSO and RSLE is the same.
The unweighted relative efficiency of PTE:
The unweighted relative efficiency of SE:
where
The unweighted relative efficiency of PRSE:
The unweighted relative efficiency of RRE:
4.5 Summary and Concluding Remarks
In this section, we discuss the contents of Tables 4.1–4.9 presented as confirmatory evidence of the theoretical findings of the estimators. First, we note that we have two classes of estimators, namely, the traditional PTE and SE and the penalty estimators. The restricted LSE plays an important role due to the fact that LASSO belongs to the class of restricted estimators.
We have the following conclusions from our study.
- Since the inception of the RRE by Hoerl and Kennard (1970), there have been articles comparing RRE with PTE and the SE. From this study, we conclude that the RRE dominates the LSE, PTE, and the SE uniformly. The PRE dominates the LASSO estimator uniformly for
greater than 0. They are 
risk equivalent at 
and at this point LASSO dominates all other estimators. The ridge estimator does not select variables but the LASSO estimator does. See Table 4.1 and graphs in Figure 4.5. - The RLSE and LASSO are
risk equivalent. Hence, LASSO satisfies “oracle properties.” - Under the family of “diagonal linear projection,” the “ideal”
risk of LASSO and subset rule (hard threshold estimator, HTE) are same and do not depend on the thresholding parameter (
) under sparse condition. SeeDonoho and Johnstone (1994). - The RWRE of estimators compared to the LSE depends upon the size of
, 
, and the divergence parameter, 
. LASSO/RLSE and RRE outperform all the estimators when 
is 0. - The LASSO satisfies the “oracle properties” and it dominates LSE, PTE, SE, and PRSE in the subinterval of
. In this case, with a small number of active parameters, the LASSO and HTE perform best followed by RRE as pointed out by Tibshirani (1996). - If
is fixed and 
increases, the RWRE of all estimators increases; see Tables 4.6 and 4.7. - If
is fixed and 
increases, the RWRE of all estimators decreases. Then, for a given 
small and 
large, the LASSO, PTE, SE, and PRSE are competitive. See Tables 4.8 and 4.9. - The PRE outperforms the LSE, PTE, SE, and PRSE uniformly. The PRE dominates LASSO and RLSE uniformly for
; and at 
, they are 
risk equivalent where 
is the divergence parameter. - The PRSE always outperforms SE; see Tables 4.1–4.9.
Table 4.1 RWRE for the estimators.
| PTE | ||||||||
 |
||||||||
 |
LSE | RLSE/LASSO |  |
 |
0.25 | SE | PRSE | RRE |
 |
||||||||
| 0 | 1 | 4.00 | 2.30 | 2.07 | 1.89 | 2.86 | 3.22 | 4.00 |
| 0.1 | 1 | 3.92 | 2.26 | 2.03 | 1.85 | 2.82 | 3.16 | 3.92 |
| 0.5 | 1 | 3.64 | 2.10 | 1.89 | 1.74 | 2.69 | 2.93 | 3.64 |
| 1 | 1 | 3.33 | 1.93 | 1.76 | 1.63 | 2.56 | 2.71 | 3.36 |
| 2 | 1 | 2.86 | 1.67 | 1.55 | 1.45 | 2.33 | 2.40 | 2.96 |
| 3 | 1 | 2.50 | 1.49 | 1.40 | 1.33 | 2.17 | 2.19 | 2.67 |
| 5 | 1 | 2.00 | 1.26 | 1.21 | 1.17 | 1.94 | 1.92 | 2.26 |
| 7 | 1 | 1.67 | 1.13 | 1.10 | 1.08 | 1.78 | 1.77 | 2.04 |
| 10 | 1 | 1.33 | 1.02 | 1.02 | 1.01 | 1.62 | 1.60 | 1.81 |
| 15 | 1 | 1.00 | 0.97 | 0.97 | 0.98 | 1.46 | 1.45 | 1.60 |
| 20 | 1 | 0.80 | 0.97 | 0.98 | 0.98 | 1.36 | 1.36 | 1.47 |
| 30 | 1 | 0.57 | 0.99 | 0.99 | 0.99 | 1.25 | 1.25 | 1.33 |
| 50 | 1 | 0.36 | 0.99 | 0.99 | 1.00 | 1.16 | 1.16 | 1.21 |
| 100 | 1 | 0.19 | 1.00 | 1.00 | 1.00 | 1.05 | 1.05 | 1.11 |
 |
||||||||
| 0 | 1 | 5.71 | 2.86 | 2.50 | 2.23 | 4.44 | 4.92 | 5.71 |
| 0.1 | 1 | 5.63 | 2.82 | 2.46 | 2.20 | 4.40 | 4.84 | 5.63 |
| 0.5 | 1 | 5.33 | 2.66 | 2.34 | 2.10 | 4.23 | 4.57 | 5.34 |
| 1 | 1 | 5.00 | 2.49 | 2.20 | 1.98 | 4.03 | 4.28 | 5.02 |
| 2 | 1 | 4.44 | 2.21 | 1.97 | 1.80 | 3.71 | 3.84 | 4.50 |
| 3 | 1 | 4.00 | 1.99 | 1.79 | 1.65 | 3.45 | 3.51 | 4.10 |
| 5 | 1 | 3.33 | 1.67 | 1.53 | 1.43 | 3.05 | 3.05 | 3.53 |
| 7 | 1 | 2.86 | 1.46 | 1.36 | 1.29 | 2.76 | 2.74 | 3.13 |
| 10 | 1 | 2.35 | 1.26 | 1.20 | 1.16 | 2.46 | 2.44 | 2.72 |
| 15 | 1 | 1.82 | 1.09 | 1.07 | 1.05 | 2.13 | 2.11 | 2.31 |
| 20 | 1 | 1.48 | 1.02 | 1.02 | 1.01 | 1.92 | 1.91 | 2.06 |
| 30 | 1 | 1.08 | 0.99 | 0.99 | 0.99 | 1.67 | 1.67 | 1.76 |
| 33 | 1 | 1.00 | 0.99 | 0.99 | 0.99 | 1.62 | 1.62 | 1.70 |
| 50 | 1 | 0.70 | 0.99 | 0.99 | 0.99 | 1.43 | 1.43 | 1.49 |
| 100 | 1 | 0.37 | 1.00 | 1.00 | 1.00 | 1.12 | 1.12 | 1.25 |
Figure 4.5 RWRE of estimates of a function of 
for 
and different 
.
Table 4.2 RWRE of the estimators for 
and different 
‐value for varying 
.
 |
 |
|||||||
| Estimators |  |
 |
 |
 |
 |
 |
 |
 |
| LSE | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
| RLSE/LASSO | 5.00 | 3.33 | 2.00 | 1.43 | 3.33 | 2.50 | 1.67 | 1.25 |
PTE ( ) |
2.34 | 1.98 | 1.51 | 1.23 | 1.75 | 1.55 | 1.27 | 1.09 |
PTE ( ) |
2.06 | 1.80 | 1.43 | 1.19 | 1.60 | 1.45 | 1.22 | 1.07 |
PTE ( ) |
1.86 | 1.66 | 1.36 | 1.16 | 1.49 | 1.37 | 1.18 | 1.06 |
| SE | 2.50 | 2.00 | 1.43 | 1.11 | 2.14 | 1.77 | 1.33 | 1.08 |
| PRSE | 3.03 | 2.31 | 1.56 | 1.16 | 2.31 | 1.88 | 1.38 | 1.10 |
| RRE | 5.00 | 3.33 | 2.00 | 1.43 | 3.46 | 2.58 | 1.71 | 1.29 |
 |
 |
|||||||
| LSE | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
| RLSE/LASSO | 1.43 | 1.25 | 1.00 | 0.83 | 0.83 | 0.77 | 0.67 | 0.59 |
PTE ( ) |
1.05 | 1.01 | 0.95 | 0.92 | 0.92 | 0.92 | 0.92 | 0.94 |
PTE ( ) |
1.03 | 1.00 | 0.95 | 0.93 | 0.94 | 0.93 | 0.94 | 0.95 |
PTE ( ) |
1.02 | 0.99 | 0.96 | 0.94 | 0.95 | 0.95 | 0.95 | 0.97 |
| SE | 1.55 | 1.38 | 1.15 | 1.03 | 1.33 | 1.22 | 1.09 | 1.01 |
| PRSE | 1.53 | 1.37 | 1.15 | 1.03 | 1.32 | 1.22 | 1.08 | 1.01 |
| RRE | 1.97 | 1.69 | 1.33 | 1.13 | 1.55 | 1.40 | 1.20 | 1.07 |
 |
 |
|||||||
| LSE | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
| RLSE/LASSO | 0.45 | 0.43 | 0.40 | 0.37 | 0.16 | 0.16 | 0.15 | 0.15 |
PTE ( ) |
0.97 | 0.97 | 0.98 | 0.99 | 1.00 | 1.00 | 1.00 | 1.00 |
PTE ( ) |
0.98 | 0.98 | 0.99 | 0.99 | 1.00 | 1.00 | 1.00 | 1.00 |
PTE ( ) |
0.98 | 0.99 | 0.99 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
| SE | 1.17 | 1.12 | 1.04 | 1.00 | 1.06 | 1.04 | 1.01 | 1.00 |
| PRSE | 1.17 | 1.12 | 1.04 | 1.00 | 1.05 | 1.04 | 1.01 | 1.00 |
| RRE | 1.30 | 1.22 | 1.11 | 1.04 | 1.10 | 1.08 | 1.04 | 1.01 |
Table 4.3 RWRE of the estimators for 
and different 
values for varying 
.
 |
 |
|||||||
| Estimators |  |
 |
 |
 |
 |
 |
 |
 |
| LSE | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
| RLSE/LASSO | 10.00 | 6.67 | 4.00 | 2.85 | 6.67 | 5.00 | 3.33 | 2.50 |
PTE ( ) |
3.20 | 2.84 | 2.31 | 1.95 | 2.50 | 2.27 | 1.93 | 1.68 |
PTE ( ) |
2.70 | 2.45 | 2.07 | 1.80 | 2.17 | 2.01 | 1.76 | 1.56 |
PTE ( ) |
2.35 | 2.17 | 1.89 | 1.67 | 1.94 | 1.82 | 1.63 | 1.47 |
| SE | 5.00 | 4.00 | 2.86 | 2.22 | 4.13 | 3.42 | 2.56 | 2.04 |
| PRSE | 6.28 | 4.77 | 3.22 | 2.43 | 4.58 | 3.72 | 2.71 | 2.13 |
| RRE | 10.00 | 6.67 | 4.00 | 2.86 | 6.78 | 5.07 | 3.37 | 2.52 |
 |
 |
|||||||
| LSE | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
| RLSE/LASSO | 2.86 | 2.50 | 2.00 | 1.67 | 1.67 | 1.54 | 1.33 | 1.18 |
PTE ( ) |
1.42 | 1.36 | 1.25 | 1.17 | 1.08 | 1.06 | 1.02 | 0.99 |
PTE ( ) |
1.33 | 1.29 | 1.20 | 1.14 | 1.06 | 1.04 | 1.02 | 0.99 |
PTE ( ) |
1.27 | 1.23 | 1.17 | 1.11 | 1.04 | 1.03 | 1.01 | 0.99 |
| SE | 2.65 | 2.36 | 1.94 | 1.65 | 2.03 | 1.87 | 1.62 | 1.43 |
| PRSE | 2.63 | 2.34 | 1.92 | 1.64 | 2.01 | 1.85 | 1.60 | 1.42 |
| RRE | 3.38 | 2.91 | 2.28 | 1.88 | 2.37 | 2.15 | 1.82 | 1.58 |
 |
 |
|||||||
| LSE | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
| RLSE/LASSO | 0.91 | 0.87 | 0.80 | 0.74 | 0.32 | 0.32 | 0.31 | 0.30 |
PTE ( ) |
0.97 | 0.97 | 0.97 | 0.97 | 1.00 | 1.00 | 1.00 | 1.00 |
PTE ( ) |
0.98 | 0.98 | 0.98 | 0.98 | 1.00 | 1.00 | 1.00 | 1.00 |
PTE ( ) |
0.99 | 0.98 | 0.98 | 0.99 | 1.00 | 1.00 | 1.00 | 1.00 |
| SE | 1.58 | 1.51 | 1.36 | 1.26 | 1.21 | 1.18 | 1.13 | 1.09 |
| PRSE | 1.58 | 1.50 | 1.36 | 1.25 | 1.21 | 1.18 | 1.13 | 1.09 |
| RRE | 1.74 | 1.64 | 1.47 | 1.34 | 1.26 | 1.23 | 1.18 | 1.13 |
Table 4.4 RWRE of the estimators for 
and different 
values for varying 
.
 |
 |
|||||||
| Estimators |  |
 |
 |
 |
 |
 |
 |
 |
| LSE | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
| RLSE/LASSO | 20.00 | 13.33 | 8.00 | 5.71 | 13.33 | 10.00 | 6.67 | 5.00 |
PTE ( ) |
4.05 | 3.74 | 3.24 | 2.86 | 3.32 | 3.12 | 2.77 | 2.49 |
PTE ( ) |
3.29 | 3.09 | 2.76 | 2.50 | 2.77 | 2.64 | 2.40 | 2.20 |
PTE ( ) |
2.78 | 2.65 | 2.42 | 2.23 | 2.40 | 2.30 | 2.13 | 1.98 |
| SE | 10.00 | 8.00 | 5.71 | 4.44 | 8.12 | 6.75 | 5.05 | 4.03 |
| PRSE | 12.80 | 9.69 | 6.52 | 4.92 | 9.25 | 7.51 | 5.45 | 4.28 |
| RRE | 20.00 | 13.33 | 8.00 | 5.71 | 13.45 | 10.07 | 6.70 | 5.02 |
 |
 |
|||||||
| LSE | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
| RLSE/LASSO | 5.71 | 5.00 | 4.00 | 3.33 | 3.33 | 3.08 | 2.67 | 2.35 |
PTE ( ) |
1.9641 | 1.8968 | 1.7758 | 1.6701 | 1.3792 | 1.3530 | 1.3044 | 1.2602 |
PTE ( ) |
1.75 | 1.70 | 1.61 | 1.53 | 1.29 | 1.27 | 1.24 | 1.20 |
PTE ( ) |
1.60 | 1.56 | 1.50 | 1.44 | 1.23 | 1.22 | 1.19 | 1.16 |
| SE | 4.87 | 4.35 | 3.59 | 3.05 | 3.46 | 3.20 | 2.78 | 2.46 |
| PRSE | 4.88 | 4.36 | 3.59 | 3.05 | 3.42 | 3.16 | 2.75 | 2.44 |
| RRE | 6.23 | 5.40 | 4.27 | 3.53 | 4.03 | 3.68 | 3.13 | 2.72 |
 |
 |
|||||||
| LSE | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
| RLSE/LASSO | 1.82 | 1.74 | 1.60 | 1.48 | 0.64 | 0.63 | 0.61 | 0.60 |
PTE ( ) |
1.05 | 1.05 | 1.03 | 1.02 | 0.99 | 0.99 | 0.99 | 0.99 |
PTE ( ) |
1.04 | 1.03 | 1.02 | 1.02 | 0.99 | 0.99 | 0.99 | 0.99 |
PTE ( ) |
1.03 | 1.02 | 1.02 | 1.01 | 0.99 | 0.99 | 1.00 | 1.00 |
| SE | 2.41 | 2.2946 | 2.09 | 1.92 | 1.52 | 1.48 | 1.42 | 1.36 |
| PRSE | 2.41 | 2.29 | 2.08 | 1.91 | 1.52 | 1.48 | 1.42 | 1.36 |
| RRE | 2.65 | 2.50 | 2.26 | 2.06 | 1.58 | 1.54 | 1.47 | 1.41 |
Table 4.5 RWRE of the estimators for 
and different 
values for varying 
.
 |
 |
|||||||
| Estimators |  |
 |
 |
 |
 |
 |
 |
 |
| LSE | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
| RLSE/LASSO | 30.00 | 20.00 | 12.00 | 8.57 | 20.00 | 15.00 | 10.00 | 7.50 |
PTE ( ) |
4.49 | 4.23 | 3.79 | 3.43 | 3.80 | 3.62 | 3.29 | 3.02 |
PTE ( ) |
3.58 | 3.42 | 3.14 | 2.91 | 3.10 | 2.99 | 2.78 | 2.59 |
PTE ( ) |
2.99 | 2.89 | 2.70 | 2.54 | 2.64 | 2.56 | 2.42 | 2.29 |
| SE | 15.00 | 12.00 | 8.57 | 6.67 | 12.12 | 10.09 | 7.55 | 6.03 |
| PRSE | 19.35 | 14.63 | 9.83 | 7.40 | 13.99 | 11.34 | 8.22 | 6.45 |
| RRE | 30.00 | 20.00 | 12.00 | 8.57 | 20.11 | 15.06 | 10.03 | 7.52 |
 |
 |
|||||||
| LSE | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
| RLSE/LASSO | 8.57 | 7.50 | 6.00 | 5.00 | 5.00 | 4.61 | 4.0000 | 3.53 |
PTE ( ) |
2.35 | 2.28 | 2.16 | 2.05 | 1.63 | 1.60 | 1.55 | 1.50 |
PTE ( ) |
2.04 | 1.99 | 1.91 | 1.83 | 1.49 | 1.47 | 1.43 | 1.39 |
PTE ( ) |
1.83 | 1.79 | 1.73 | 1.67 | 1.39 | 1.37 | 1.34 | 1.31 |
| SE | 7.10 | 6.35 | 5.25 | 4.47 | 4.89 | 4.53 | 3.94 | 3.50 |
| PRSE | 7.17 | 6.41 | 5.28 | 4.50 | 4.84 | 4.48 | 3.91 | 3.47 |
| RRE | 9.09 | 7.90 | 6.26 | 5.19 | 5.70 | 5.21 | 4.45 | 3.89 |
 |
 |
|||||||
| LSE | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
| RLSE/LASSO | 2.73 | 2.61 | 2.40 | 2.22 | 0.97 | 0.95 | 0.92 | 0.89 |
PTE ( ) |
1.15 | 1.14 | 1.13 | 1.11 | 0.99 | 0.99 | 0.99 | 0.99 |
PTE ( ) |
1.11 | 1.10 | 1.09 | 1.08 | 0.99 | 0.99 | 0.99 | 0.99 |
PTE ( ) |
1.08 | 1.08 | 1.07 | 1.06 | 0.99 | 0.99 | 0.99 | 0.99 |
| SE | 3.25 | 3.09 | 2.82 | 2.60 | 1.83 | 1.79 | 1.72 | 1.65 |
| PRSE | 3.23 | 3.08 | 2.81 | 2.59 | 1.83 | 1.79 | 1.72 | 1.65 |
| RRE | 3.55 | 3.37 | 3.05 | 2.79 | 1.90 | 1.86 | 1.78 | 1.71 |
Table 4.6 RWRE values of estimators for 
and different values of 
and 
.
| PTE | ||||||||
 |
||||||||
| LSE | RLSE/LASSO |  |
 |
0.25 | SE | PRSE | RRE | |
 |
 |
|||||||
| 5 | 1.00 | 2.00 | 1.76 | 1.51 | 1.36 | 1.43 | 1.56 | 2.00 |
| 15 | 1.00 | 4.00 | 3.11 | 2.31 | 1.89 | 2.86 | 3.22 | 4.00 |
| 25 | 1.00 | 6.00 | 4.23 | 2.84 | 2.20 | 4.28 | 4.87 | 6.00 |
| 35 | 1.00 | 8.00 | 5.18 | 3.24 | 2.42 | 5.71 | 6.52 | 8.00 |
| 55 | 1.00 | 12.00 | 6.71 | 3.79 | 2.70 | 8.57 | 9.83 | 12.00 |
 |
 |
|||||||
| 5 | 1.00 | 1.82 | 1.58 | 1.37 | 1.26 | 1.37 | 1.46 | 1.83 |
| 15 | 1.00 | 3.64 | 2.79 | 2.10 | 1.74 | 2.70 | 2.93 | 3.65 |
| 25 | 1.00 | 5.45 | 3.81 | 2.61 | 2.05 | 4.03 | 4.43 | 5.46 |
| 35 | 1.00 | 7.27 | 4.68 | 2.98 | 2.26 | 5.36 | 5.93 | 7.28 |
| 55 | 1.00 | 10.91 | 6.11 | 3.52 | 2.55 | 8.02 | 8.94 | 10.92 |
 |
 |
|||||||
| 5 | 1.00 | 1.67 | 1.43 | 1.27 | 1.18 | 1.33 | 1.38 | 1.71 |
| 15 | 1.00 | 3.33 | 2.53 | 1.93 | 1.63 | 2.56 | 2.71 | 3.37 |
| 25 | 1.00 | 5.00 | 3.46 | 2.41 | 1.92 | 3.80 | 4.08 | 5.03 |
| 35 | 1.00 | 6.67 | 4.27 | 2.77 | 2.13 | 5.05 | 5.45 | 6.70 |
| 55 | 1.00 | 10.00 | 5.61 | 3.29 | 2.42 | 7.55 | 8.22 | 10.03 |
 |
 |
|||||||
| 5 | 1.0000 | 1.00 | 0.93 | 0.95 | 0.96 | 1.15 | 1.15 | 1.33 |
| 15 | 1.00 | 2.00 | 1.47 | 1.26 | 1.17 | 1.94 | 1.92 | 2.28 |
| 25 | 1.00 | 3.00 | 1.98 | 1.54 | 1.35 | 2.76 | 2.75 | 3.27 |
| 35 | 1.00 | 4.00 | 2.44 | 1.77 | 1.50 | 3.59 | 3.59 | 4.27 |
| 55 | 1.00 | 6.00 | 3.27 | 2.16 | 1.73 | 5.25 | 5.28 | 6.26 |
Table 4.7 RWRE values of estimators for 
and different values of 
and 
.
 |
 |
|||||||
| 5 | 1.00 | 1.43 | 1.33 | 1.23 | 1.16 | 1.11 | 1.16 | 1.43 |
| 15 | 1.00 | 2.86 | 2.41 | 1.94 | 1.67 | 2.22 | 2.43 | 2.86 |
| 25 | 1.00 | 4.28 | 3.35 | 2.46 | 2.00 | 3.33 | 3.67 | 4.28 |
| 35 | 1.00 | 5.71 | 4.17 | 2.86 | 2.23 | 4.44 | 4.92 | 5.71 |
| 55 | 1.00 | 8.57 | 5.54 | 3.43 | 2.53 | 6.67 | 7.40 | 8.57 |
 |
 |
|||||||
| 5 | 1.00 | 1.33 | 1.23 | 1.15 | 1.10 | 1.09 | 1.13 | 1.35 |
| 15 | 1.00 | 2.67 | 2.22 | 1.80 | 1.56 | 2.12 | 2.27 | 2.67 |
| 25 | 1.00 | 4.00 | 3.08 | 2.29 | 1.87 | 3.17 | 3.41 | 4.00 |
| 35 | 1.00 | 5.33 | 3.84 | 2.66 | 2.10 | 4.23 | 4.57 | 5.34 |
| 55 | 1.00 | 8.00 | 5.13 | 3.21 | 2.40 | 6.33 | 6.89 | 8.00 |
 |
 |
|||||||
| 5 | 1.00 | 1.25 | 1.15 | 1.09 | 1.06 | 1.08 | 1.10 | 1.29 |
| 15 | 1.00 | 2.50 | 2.05 | 1.68 | 1.47 | 2.04 | 2.13 | 2.52 |
| 25 | 1.00 | 3.75 | 2.85 | 2.13 | 1.77 | 3.03 | 3.20 | 3.77 |
| 35 | 1.00 | 5.00 | 3.56 | 2.49 | 1.98 | 4.03 | 4.28 | 5.01 |
| 55 | 1.00 | 7.50 | 4.77 | 3.02 | 2.29 | 6.03 | 6.45 | 7.52 |
 |
 |
|||||||
| 5 | 1.00 | 0.83 | 0.87 | 0.92 | 0.94 | 1.03 | 1.03 | 1.13 |
| 15 | 1.00 | 1.67 | 1.32 | 1.17 | 1.11 | 1.65 | 1.64 | 1.88 |
| 25 | 1.00 | 2.50 | 1.78 | 1.44 | 1.29 | 2.34 | 2.34 | 2.70 |
| 35 | 1.00 | 3.33 | 2.20 | 1.67 | 1.44 | 3.05 | 3.05 | 3.53 |
| 55 | 1.00 | 5.00 | 2.98 | 2.05 | 1.67 | 4.47 | 4.50 | 5.19 |
Table 4.8 RWRE values of estimators for 
and different values of 
and 
.
| PTE | ||||||||
 |
||||||||
| LSE | RLSE/LASSO |  |
 |
0.25 | SE | PRSE | RRE | |
 |
 |
|||||||
| 5 | 1.00 | 2.00 | 1.76 | 1.51 | 1.36 | 1.43 | 1.56 | 2.00 |
| 15 | 1.00 | 1.33 | 1.27 | 1.20 | 1.15 | 1.18 | 1.22 | 1.33 |
| 25 | 1.00 | 1.20 | 1.17 | 1.127 | 1.10 | 1.11 | 1.14 | 1.20 |
| 35 | 1.00 | 1.14 | 1.12 | 1.09 | 1.07 | 1.08 | 1.10 | 1.14 |
| 55 | 1.00 | 1.09 | 1.08 | 1.06 | 1.04 | 1.05 | 1.06 | 1.09 |
 |
 |
|||||||
| 5 | 1.00 | 1.82 | 1.58 | 1.37 | 1.26 | 1.34 | 1.46 | 1.83 |
| 15 | 1.00 | 1.29 | 1.22 | 1.16 | 1.11 | 1.16 | 1.19 | 1.29 |
| 25 | 1.00 | 1.18 | 1.14 | 1.10 | 1.07 | 1.10 | 1.12 | 1.18 |
| 35 | 1.00 | 1.13 | 1.10 | 1.07 | 1.05 | 1.07 | 1.08 | 1.13 |
| 55 | 1.00 | 1.08 | 1.06 | 1.05 | 1.03 | 1.05 | 1.05 | 1.08 |
 |
 |
|||||||
| 5 | 1.00 | 1.67 | 1.43 | 1.27 | 1.18 | 1.33 | 1.38 | 1.71 |
| 15 | 1.00 | 1.25 | 1.18 | 1.12 | 1.08 | 1.14 | 1.16 | 1.26 |
| 25 | 1.00 | 1.15 | 1.11 | 1.08 | 1.05 | 1.09 | 1.10 | 1.16 |
| 35 | 1.00 | 1.11 | 1.08 | 1.06 | 1.04 | 1.07 | 1.07 | 1.12 |
| 55 | 1.00 | 1.07 | 1.05 | 1.04 | 1.03 | 1.04 | 1.05 | 1.07 |
 |
 |
|||||||
| 5 | 1.00 | 1.00 | 0.93 | 0.95 | 0.96 | 1.15 | 1.15 | 1.33 |
| 15 | 1.00 | 1.00 | 0.97 | 0.97 | 0.98 | 1.07 | 1.07 | 1.14 |
| 25 | 1.00 | 1.00 | 0.98 | 0.98 | 0.98 | 1.05 | 1.04 | 1.09 |
| 35 | 1.00 | 1.00 | 0.98 | 0.99 | 0.99 | 1.03 | 1.03 | 1.07 |
| 55 | 1.00 | 1.00 | 0.99 | 0.99 | 0.99 | 1.02 | 1.02 | 1.04 |
Table 4.9 RWRE values of estimators for 
and different values of 
and 
.
| PTE | ||||||||
 |
||||||||
| LSE | RLSE/LASSO |  |
 |
0.25 | SE | PRSE | RRE | |
 |
 |
|||||||
| 3 | 1.00 | 3.33 | 2.60 | 1.98 | 1.66 | 2.00 | 2.31 | 3.33 |
| 13 | 1.00 | 1.54 | 1.44 | 1.33 | 1.24 | 1.33 | 1.40 | 1.54 |
| 23 | 1.00 | 1.30 | 1.26 | 1.20 | 1.15 | 1.20 | 1.23 | 1.30 |
| 33 | 1.00 | 1.21 | 1.18 | 1.14 | 1.11 | 1.14 | 1.16 | 1.21 |
| 53 | 1.00 | 1.13 | 1.11 | 1.09 | 1.07 | 1.09 | 1.10 | 1.13 |
 |
 |
|||||||
| 3 | 1.00 | 2.86 | 2.21 | 1.73 | 1.49 | 1.87 | 2.06 | 2.88 |
| 13 | 1.00 | 1.48 | 1.38 | 1.27 | 1.20 | 1.30 | 1.35 | 1.48 |
| 23 | 1.00 | 1.28 | 1.22 | 1.16 | 1.12 | 1.18 | 1.20 | 1.28 |
| 33 | 1.00 | 1.19 | 1.16 | 1.12 | 1.09 | 1.13 | 1.15 | 1.19 |
| 53 | 1.00 | 1.12 | 1.10 | 1.07 | 1.06 | 1.08 | 1.09 | 1.12 |
 |
 |
|||||||
| 3 | 1.00 | 2.50 | 1.93 | 1.55 | 1.37 | 1.77 | 1.88 | 2.58 |
| 13 | 1.00 | 1.43 | 1.32 | 1.22 | 1.16 | 1.28 | 1.31 | 1.44 |
| 23 | 1.00 | 1.25 | 1.19 | 1.13 | 1.10 | 1.17 | 1.18 | 1.26 |
| 33 | 1.00 | 1.18 | 1.14 | 1.10 | 1.07 | 1.12 | 1.13 | 1.18 |
| 53 | 1.00 | 1.11 | 1.09 | 1.06 | 1.05 | 1.08 | 1.08 | 1.11 |
 |
 |
|||||||
| 3 | 1.00 | 1.25 | 1.04 | 1.01 | 0.99 | 1.38 | 1.372 | 1.69 |
| 13 | 1.00 | 1.11 | 1.02 | 1.00 | 0.99 | 1.16 | 1.15 | 1.26 |
| 23 | 1.00 | 1.07 | 1.01 | 1.00 | 0.99 | 1.10 | 1.10 | 1.16 |
| 33 | 1.00 | 1.05 | 1.01 | 1.00 | 0.99 | 1.07 | 1.07 | 1.11 |
| 53 | 1.00 | 1.03 | 1.01 | 1.00 | 0.99 | 1.05 | 1.05 | 1.07 |
Problems
- 4.1 Show that the test statistic for testing
vs. 
for unknown 
is
and also show that
has a noncentral 
with appropriate DF and noncentrality parameter 
. - 4.2
Determine the bias vector of estimators,
and 
in Eqs. (4.15) and (4.17), respectively. - 4.3 Show that the bias and MSE of
are, respectively,
(4.46)
and
(4.47)
- 4.4 Prove under usual notation that the RRE uniformly dominates both LSE and PTEs.
- 4.5 Verify that RRE uniformly dominates both Stein‐type and its positive‐rule estimators.
- 4.6 Prove under usual notation that the RRE uniformly dominates LASSO.
- 4.7
Show that the modified LASSO outperforms the SE as well as the PRSE in the interval
