In our mind, the primary issue to be decided prior to a higher-order factor analysis is whether the initial factor structure is appropriate or not. If we extract five factors, and then decide there is a single second-order factor (or even two second-order factors), the first question we would ask is whether the original solution was correct, or whether the correct first-order structure should have been one (or two) factors rather than five.[2]

In the seven or eight decades since this discussion began in earnest, many things have changed in quantitative methods. One is the easy access to confirmatory factor analysis techniques. Although we have not discussed confirmatory techniques, they are methods for directly testing hypotheses such as whether a particular data set is best characterized as one, two, or five factors. Thus, before launching into higher-order factor analysis, we would evaluate (and replicate) whether the initial solution was correct. We suspect that, in many cases, the initial solution was indefensible or suboptimal, and that the “higher-order factors” are really just the more parsimonious version of what should have been extracted initially. We would recommend exploring higher-order factors only after the initial factor structure has been thoroughly vetted through CFA as the most parsimonious and desirable. Of course, once in CFA, higher-order factors can be modeled and tested in that confirmatory framework![3]

If you want to explore this aspect of your data, we can, in the spirit of intrepid exploration, briefly cover some of the mechanics of the process.