Application: Factor Score Estimation

Now, let’s walk through an example to demonstrate some of the challenges and issues associated with the estimation of factor scores. We use the following syntax to run our factor analysis for the engineering data, to output a data set called factor_scores1 containing the predefined SAS factor scores (based on the standardized scoring coefficients) for each individual, to output a data set called Results that contains the pattern loading and standardized scoring coefficients along with other summary results, and to output a table containing the standardized scoring coefficients. The two output data sets and the summary table were requested by adding the OUT=, OUTSTAT=, and SCORE options to the FACTOR step, as described above.
proc factor data = engdata  nfactors = 2  method = prinit  priors = SMC 
      rotate = oblimin out=factor_scores1 outstat=results score;
   var EngProb: INTERESTeng: ;
run;
The pattern loading matrix and the standardized scoring coefficients output by SAS are presented in Pattern coefficients and standardized score coefficients from engineering data. These two sets of coefficients represent different potential weights that could be used to estimate factor scores and that we will use for the current example. We include the pattern matrix because you are likely to be more familiar with it, and we present the standardized scoring coefficients as a representation of the built-in SAS weights for computing factor scores.
Table 9.1 Pattern coefficients and standardized score coefficients from engineering data
Var:
Rotated pattern coefficients
Standardized scoring coefficients
1
2
1
2
EngProbSolv1
.859
-.016
.142
-.013
EngProbSolv2
.841
-.071
.091
-.010
EngProbSolv3
.879
-.008
.147
.010
EngProbSolv4
.909
-.025
.187
-.006
EngProbSolv5
.886
.021
.158
.006
EngProbSolv6
.869
.020
.140
.008
EngProbSolv7
.868
.033
.151
.021
EngProbSolv8
.790
.072
.084
.009
INTERESTeng1
.042
.801
.003
.086
INTERESTeng2
-.023
.921
.001
.225
INTERESTeng3
-.014
.922
-.004
.233
INTERESTeng4
-.001
.904
.012
.178
INTERESTeng5
-.007
.897
.001
.209
INTERESTeng6
.009
.864
.002
.131
The syntax provided above will output a data set containing factor scores for each individual in the data set based on the predefined SAS method of computing scores, which in turn is based on the standardized scoring coefficients. However, we must do some additional work to compute factor scores based on the pattern matrix. This syntax, along with explanatory notes, is provided below.
*Grab and restructure pattern loadings;
data coeff_oneline (keep=F1_: F2_: merge);
   set results end=last;
   *Array of new variables to contain all loadings in one line;
   array newVar(2,14) F1_EngProbSolv1-F1_EngProbSolv8 F1_INTERESTeng1-
      F1_INTERESTeng6 F2_EngProbSolv1-F2_EngProbSolv8 F2_INTERESTeng1-
      F2_INTERESTeng6;
   *Array of variables containing pattern loadings in input data set;
   array oldVar(*) EngProbSolv1-EngProbSolv8 INTERESTeng1-INTERESTeng6;
   *Retain statement to tell SAS to keep values assigned into these 
      variables across records so that the last record will contain all 
      values assigned;
   retain F1_EngProbSolv1-F1_EngProbSolv8 F1_INTERESTeng1-F1_INTERESTeng6 
      F2_EngProbSolv1-F2_EngProbSolv8 F2_INTERESTeng1-F2_INTERESTeng6;
   *Identify pattern matrix loadings and assign them to the new 
      variables;
   if _TYPE_='PATTERN' and _NAME_='Factor1' then do i=1 to 14;
      newVar(1,i)=oldVar(i);
   end;
   if _TYPE_='PATTERN' and _NAME_='Factor2' then do i=1 to 14;
      newVar(2,i)=oldVar(i);
   end;
   merge=1; *add merge variable so we can merge the resulting one line 
      with all records in student data set;
   *Output the last record in the data set that will contain all the new 
   variables;
   if last then output;
run;
*Assign merge variable to data set containing student records and pre-
   defined SAS factor scores so we can merge the above pattern loadings 
   with each record and compute factor scores based on pattern loadings;
data factor_scores1;
   set factor_scores1;
   merge=1; *add merge variable;
run;
*Compute factor scores based on pattern loadings;
data pattern_score_factorScores (drop=merge F1_: F2_: wt: i f);
   merge factor_scores1 coeff_oneline;
   by merge;
   *Array of variables containing pattern loadings;
   array fact(2,14) F1_EngProbSolv1-F1_EngProbSolv8 F1_INTERESTeng1-
      F1_INTERESTeng6 F2_EngProbSolv1-F2_EngProbSolv8 F2_INTERESTeng1-
      F2_INTERESTeng6;
*Array of responses to items;
   array itms(*) EngProbSolv1-EngProbSolv8 INTERESTeng1-INTERESTeng6;
   *Array that will contain weighted item scores (pattern loading x response);
   array wtItm(*) wt1-wt14;
   *Array that will contain final factor scores based on pattern loadings;
   array fs(*) patternF1 patternF2;
   *Loop from 1 to 2 – representing the factors;
   do f=1 to 2;
      *Loop through each of the 14 items to compute weighted item scores;
      do i=1 to dim(itms);
         wtItm(i)=fact(f,i)*itms(i);
      end;
      *Compute mean of the weighted item scores as the factor score;
      fs(f)=mean(of wt:); *compute mean of weighted estimates as the 
         factor score;
   end;
run;
Methods of factor weighting. Many different types of weights are used in the estimation of factor scores. As we mentioned above, Pattern coefficients and standardized score coefficients from engineering data displays two such types that can be used to estimate factor scores for the engineering data. Let’s do a quick comparison of these weights to determine the comparability of their subsequent factor scores. An initial glance will tell you that the coefficients are in a different metric. Regardless of whether you sum or average the different composites taken, the scores will be on completely different scales. Nevertheless, the coefficients do tend to follow a similar rank order. A few of the coefficients deviate from this pattern, but the deviations are minor in size. Furthermore, if we examine the correlation between the two sets of factors scores produced (see Figure 9.2 Correlations between sets of factor scores estimated from the engineering data), we would conclude that the methods provide the same fundamental results. Thus, at least when it comes to picking between different weighted methods, the specific weights used might not make a large difference in the results.
Figure 9.2 Correlations between sets of factor scores estimated from the engineering data
Proper vs improper factor scores. Researchers must also decide whether they will use all of the variables to compute a factor score (a proper factor score) or just the variables that compose a particular factor (i.e., those measured variables with the highest factor loadings on a particular factor). Proper factor scores take into account all variables in the analysis, not just the ones that are considered to be part of a factor. The scores automatically produced by SAS (using the OUT option) are proper factor scores. So, referring to Pattern coefficients and standardized score coefficients from engineering data, a proper factor score for the first factor (using either set of coefficients) would include both the engineering problem-solving variables and the identification with engineering variables. Of course, those second loadings are relatively small and, as such, would contribute very little to the analysis. An improper factor score would include only those variables considered to be part of the factor (e.g., only EngProbSolv1 – EngProbSolv8 would be used to compute the factor 1 score). In practice, these two types of factor scores are generally well correlated; yet in our opinion, both types are equally flawed.
Stability of factor scores. It is arguable whether weighted factor scores are actually an improvement over equally weighted composites (averaging, summing). On one hand, weighted factor scores do account for the fact that some items contribute much more strongly to constructs than others. On the other hand, if we are concerned about replicability, and if factor loadings are volatile and not likely to be replicable, we could be causing more harm than good by using this approach.
Let us examine the replicability of the pattern matrix coefficients across four random samples from the engineering data. We review only the pattern matrix coefficients among these samples because we are more familiar with them and, as we discussed above, the different weighting methods yield similar results. Thus, our findings with the pattern coefficients can likely be extended to other weighted factor scores. We have already established that smaller samples have more variability than large samples, so let us examine a few random samples at the 10:1 (participant: item) ratio (N= 140).[5]
Table 9.2 Variability in engineering data pattern coefficients across four random samples
Var:
Sample 1
Sample 2
Sample 3
Sample 4
1
2
1
2
1
2
1
2
EngProbSolv1
.82
.01
.88
.00
.83
.01
.90
-.04
EngProbSolv2
.79
-.06
.82
.00
.84
-.06
.87
-.09
EngProbSolv3
.87
-.02
.92
.00
.87
.02
.91
.01
EngProbSolv4
.88
-.04
.95
-.06
.92
.00
.91
.00
EngProbSolv5
.84
.03
.91
-.02
.90
.01
.87
.04
EngProbSolv6
.81
.05
.90
-.01
.85
-.04
.90
.03
EngProbSolv7
.81
.01
.88
.05
.85
.01
.85
.07
EngProbSolv8
.80
.05
.84
.04
.76
.08
.84
.02
INTERESTeng1
.21
.66
-.01
.87
-.08
.85
-.01
.89
INTERESTeng2
-.07
.92
.01
.93
.03
.84
.01
.92
INTERESTeng3
-.09
.94
-.01
.95
.01
.88
.01
.92
INTERESTeng4
-.10
.95
.01
.95
.09
.78
.04
.83
INTERESTeng5
.11
.80
.00
.93
-.03
.87
-.04
.92
INTERESTeng6
.04
.75
.01
.88
.00
.84
.01
.91
Note: Primary factors each variable loads on are highlighted
The pattern matrices for our samples are presented in Variability in engineering data pattern coefficients across four random samples. Examining these four small random samples, you should see enough variability to encourage you to be skeptical of the concept of weighted factor scores. For example, the first item in engineering problem-solving ranges from 0.66 to 0.89 across the four samples. Several other items display similar ranges. If you consider these as correlation coefficients (for a simple example), these represent a substantial range of variance accounted for (43.56% vs 79.21%). Of course, these are interpreted in a more complex manner than a simple correlation coefficient, but our point is that these are not precise (nor reliable) estimates of the population parameter and, as such, should give you pause. Remember that this is a scale that had very strong psychometric properties.
If you find yourself unconvinced, review the previous chapter on bootstrap resampling and examine the replicability of the pattern matrices. The engineering scale was rather solid and stable, with relatively narrow confidence intervals around the pattern loadings. Although, even these narrow confidence intervals contain more variability than desirable when estimating factor scores. A similarly sized sample from the SDQ data set exhibited even larger confidence intervals around its coefficients. Thus, pattern loadings and subsequent factor scores can vary more among scales with a less clear structure. You can also refer to Grice’s (2001) excellent work on the topic for more information.
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset