Confirmatory Factor Analysis
Methods. Using the second group of randomly selected respondents, we conducted CFAs using the ten EIs organized by four primary content area themes: Academic Challenge, Learning with Peers, Experiences with Faculty, and Campus Environment. We developed separate models for all first-year students, all seniors, online first-year students, and online seniors, including conceptually related EIs together in the same model and allowing them to correlate.
Given its larger size, we used the senior sample to develop an initial set of four models with the EIs grouped into the conceptually related areas. After building these models, we used modification indices to determine whether model fit could be improved by correlating the error terms of factor indicators (or individual survey items). Once the final models with the greatest number of paths for each content area had been created using the senior population, we then proceeded by estimating model fit indices, standardized regression weights, and factor correlation estimates for first-year student, online first-year student, and online senior populations.
To assess model fit, we considered five different indices: CMIN/DF (chi-square divided by degrees of freedom), GFI (goodness of fit index), CFI (comparative fit index), RMSEA (root mean square error of approximation), and PCLOSE (p-value for test of close fit). Traditional good model fit criteria for CMIN/DF is a value of 5 or less; however, this statistic is very sensitive to sample size and likely to be inflated with large samples. For the other fit indices, good model fit criteria (as recommended by Hu & Bentler, 1999) are as follows:
GFI: .85 or higher
CFI: .90 or higher
RMSEA: .06 or lower
PCLOSE: .05 or higher
Standardized regression weights determined the strength of factor loadings. These values could range between 0 and 1, with higher values being more desirable. We considered values under .40 unacceptably low (Kline, 2002). Correlations between content area factors of .80 or greater indicate multicollinearity and the potential for factor indicators to load on more than one factor.