Online appendix for Pornprasertmanit, Sriutaisuk, Heene, and Wu (in preparation). “Equivalence testing of expected parameter changes: A new way to assess model fit in structural equation modeling”.
This online appendix provides a step-by-step illustration of equivalence testing based on expected parameter changes (EPCs) in a confirmatory factor analysis (CFA) model. We demonstrate how EPC-based equivalence testing can be implemented in practice, how to interpret both global and local decisions, and how model evaluation depends jointly on the current model specification and the prespecified smallest effect size of interest (SESOI).
In addition, we illustrate the use of the diagnostic function
epcEquivCheck(), which evaluates whether EPC-based
equivalence testing may yield large EPCs even when the underlying
misspecifications are relatively small—a phenomenon referred to as the
compensatory effect. When the compensatory effect is
pronounced, large EPCs may reflect parameter compensation rather than
genuine substantive misspecification. In such cases, model adjustments
must be interpreted with particular caution.
All models are fit to the same correlation matrix using a fixed
sample size of \(N=4,000\). EPC
equivalence testing is conducted using epcEquivFit(), and
compensatory effects are assessed using epcEquivCheck().
Both functions are available in the semTools package
(version 0.5-8 or later), with model estimation carried out in
lavaan.
library(lavaan)
library(semTools)Christopher et al. (2012) examined relations among reading and comprehension abilities in children. The population correlation matrix for the age 8–10 group was extracted from Table 2 of the original article and is reproduced here for illustration purposes.
cor_mat <- matrix(
c(
1.00, 0.44, 0.54, 0.58, 0.44, 0.48, 0.49, 0.42, 0.36, 0.42,
0.44, 1.00, 0.44, 0.47, 0.60, 0.40, 0.40, 0.32, 0.27, 0.33,
0.54, 0.44, 1.00, 0.45, 0.48, 0.45, 0.39, 0.27, 0.17, 0.25,
0.58, 0.47, 0.45, 1.00, 0.52, 0.47, 0.66, 0.68, 0.61, 0.72,
0.44, 0.60, 0.48, 0.52, 1.00, 0.34, 0.51, 0.42, 0.36, 0.40,
0.48, 0.40, 0.45, 0.47, 0.34, 1.00, 0.42, 0.34, 0.33, 0.33,
0.49, 0.40, 0.39, 0.66, 0.51, 0.42, 1.00, 0.74, 0.66, 0.69,
0.42, 0.32, 0.27, 0.68, 0.42, 0.34, 0.74, 1.00, 0.75, 0.85,
0.36, 0.27, 0.17, 0.61, 0.36, 0.33, 0.66, 0.75, 1.00, 0.73,
0.42, 0.33, 0.25, 0.72, 0.40, 0.33, 0.69, 0.85, 0.73, 1.00
),
nrow = 10,
byrow = TRUE,
dimnames = list(
c("WJORAL","QRLL","Barnes","WJPC","QRIR","GORT",
"PIATC","PIATR","PIATS","WordR"),
c("WJORAL","QRLL","Barnes","WJPC","QRIR","GORT",
"PIATC","PIATR","PIATS","WordR")
)
)Variable abbreviations are as follows:
Because EPC-based equivalence testing relies on sufficiently narrow confidence intervals (CIs) for EPCs, the sample size is fixed at \(N=4,000\) to ensure adequate precision. This choice is made solely for illustrative purposes and should not be interpreted as a recommendation for applied research.
N <- 4000The baseline model corresponds to Figure 2 in the original article
(see also Figure X2 in the present manuscript). The model specifies two
correlated latent factors, comprehension and word
reading, with a single residual covariance between
QRLL and QRIR. Two indicators
(WJPC and PIATC) load on both factors.
model_base <- '
comp =~ WJORAL + QRLL + Barnes + GORT + QRIR + WJPC + PIATC
wordreading =~ WJPC + PIATC + PIATR + PIATS + WordR
QRLL ~~ QRIR
'Fit the model to the sample correlation defined above.
fit_base <- cfa(model_base, sample.cov = cor_mat, sample.nobs = N)
summary(fit_base, std = TRUE, fit = TRUE)## lavaan 0.6-21 ended normally after 33 iterations
##
## Estimator ML
## Optimization method NLMINB
## Number of model parameters 24
##
## Number of observations 4000
##
## Model Test User Model:
##
## Test statistic 791.378
## Degrees of freedom 31
## P-value (Chi-square) 0.000
##
## Model Test Baseline Model:
##
## Test statistic 25007.308
## Degrees of freedom 45
## P-value 0.000
##
## User Model versus Baseline Model:
##
## Comparative Fit Index (CFI) 0.970
## Tucker-Lewis Index (TLI) 0.956
##
## Loglikelihood and Information Criteria:
##
## Loglikelihood user model (H0) -44644.576
## Loglikelihood unrestricted model (H1) -44248.887
##
## Akaike (AIC) 89337.152
## Bayesian (BIC) 89488.209
## Sample-size adjusted Bayesian (SABIC) 89411.947
##
## Root Mean Square Error of Approximation:
##
## RMSEA 0.078
## 90 Percent confidence interval - lower 0.074
## 90 Percent confidence interval - upper 0.083
## P-value H_0: RMSEA <= 0.050 0.000
## P-value H_0: RMSEA >= 0.080 0.284
##
## Standardized Root Mean Square Residual:
##
## SRMR 0.041
##
## Parameter Estimates:
##
## Standard errors Standard
## Information Expected
## Information saturated (h1) model Structured
##
## Latent Variables:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## comp =~
## WJORAL 1.000 0.754 0.754
## QRLL 0.822 0.023 36.025 0.000 0.620 0.620
## Barnes 0.916 0.023 40.418 0.000 0.690 0.690
## GORT 0.833 0.023 36.817 0.000 0.628 0.628
## QRIR 0.859 0.023 37.717 0.000 0.648 0.648
## WJPC 0.607 0.020 30.065 0.000 0.458 0.458
## PIATC 0.397 0.019 20.659 0.000 0.299 0.299
## wordreading =~
## WJPC 1.000 0.496 0.496
## PIATC 1.234 0.038 32.564 0.000 0.613 0.613
## PIATR 1.871 0.049 37.814 0.000 0.929 0.929
## PIATS 1.629 0.046 35.566 0.000 0.809 0.809
## WordR 1.840 0.049 37.624 0.000 0.913 0.913
##
## Covariances:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## .QRLL ~~
## .QRIR 0.198 0.012 16.642 0.000 0.198 0.332
## comp ~~
## wordreading 0.210 0.009 22.663 0.000 0.561 0.561
##
## Variances:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## .WJORAL 0.432 0.013 33.338 0.000 0.432 0.432
## .QRLL 0.616 0.016 38.966 0.000 0.616 0.616
## .Barnes 0.523 0.014 36.895 0.000 0.523 0.523
## .GORT 0.605 0.015 39.163 0.000 0.605 0.605
## .QRIR 0.580 0.015 38.169 0.000 0.580 0.580
## .WJPC 0.289 0.008 36.516 0.000 0.289 0.289
## .PIATC 0.329 0.008 39.583 0.000 0.329 0.329
## .PIATR 0.137 0.005 25.165 0.000 0.137 0.137
## .PIATS 0.346 0.009 39.105 0.000 0.346 0.346
## .WordR 0.166 0.006 28.679 0.000 0.166 0.166
## comp 0.568 0.022 25.901 0.000 1.000 1.000
## wordreading 0.246 0.014 18.008 0.000 1.000 1.000Model fit is mixed relative to commonly used cutoff criteria (Hu
& Bentler, 1999). Although incremental fit indices (CFI and TLI) are
acceptable, RMSEA exceeds conventional thresholds. All standardized
factor loadings are substantial except the loading of the comprehension
factor on PIATC (.299), which falls slightly below commonly
cited guidelines (.30–.40).
We next apply EPC-based equivalence testing using
epcEquivFit(). The SESOI is set to .40 for standardized
factor loadings and .10 for residual correlations.
epc_base <- epcEquivFit(fit_base, stdLoad = 0.4, cor = 0.1)
summary(epc_base)## Global EPC Evaluation Summary
## --------------------------------
##
## [1. EPC Equivalence Testing: CI-based]
## Substantially Misspecified (M): 7
## Inconclusive (I): 10
## CI-Underpowered (U): 0
## Trivial / Not Misspecified (NM): 35
##
## Global EPC Equivalence Decision: SUBSTANTIAL MISSPECIFICATION
##
## 1.1 Top Substantially Misspecified EPCs (ranked by |std.epc / SESOI|):
## lhs op rhs std.epc std.sesoi severity
## 72 WJPC ~~ WordR 0.286 0.1 2.863
## 70 WJPC ~~ PIATR -0.240 0.1 2.396
## 75 PIATC ~~ WordR -0.221 0.1 2.205
## 73 PIATC ~~ PIATR 0.179 0.1 1.789
## 56 Barnes ~~ PIATS -0.174 0.1 1.742
##
## 1.2 Top CI-inconclusive EPCs
## (ranked by distance to equivalence bounds; larger = needs narrower CI):
## lhs op rhs lower.std.epc upper.std.epc ci_gap
## 77 PIATR ~~ WordR 0.025 0.186 0.075
## 78 PIATS ~~ WordR -0.136 -0.052 0.036
## 62 GORT ~~ PIATS 0.067 0.126 0.026
## 36 WJORAL ~~ Barnes 0.048 0.124 0.024
## 69 WJPC ~~ PIATC -0.140 -0.077 0.023The EPC evaluation indicates no underpowered fixed parameters. However, seven fixed parameters are classified as substantially misspecified, yielding a global decision of substantial misspecification. Ten fixed parameters are inconclusive, and the remaining 35 are classified as trivially misspecified.
Inspection of the top substantially misspecified EPCs reveals that
the largest misspecification corresponds to the residual covariance
between WJPC and WordR. Several additional
residual correlations also appear among the most substantially
misspecified EPCs, suggesting that the baseline model omits
substantively meaningful local dependencies.
Overall, under the specified SESOI, the baseline model cannot be regarded as approximately correct.
The function epcEquivCheck() evaluates whether
substantial misspecifications might be driven by inflated EPCs due to
the compensatory effect. Each fixed parameter is perturbed by \(\pm 75\%\) of the SESOI. If any resulting
EPC exceeds the SESOI in magnitude, the compensatory effect is
considered pronounced, and substantial misspecifications must be
interpreted cautiously.
epcEquivCheck(fit_base)## EPC Equivalence Compensatory Effect (Standardized Parameters)
## ----------------------------------------------------
## Feasible standardized population: TRUE
## EPCs exceeding SESOI under tested perturbations (summary):
## perturbed_lhs perturbed_op perturbed_rhs resulting_lhs resulting_op
## 1 wordreading =~ WJORAL wordreading =~
## 2 wordreading =~ Barnes wordreading =~
## 3 wordreading =~ QRIR wordreading =~
## 4 wordreading =~ QRLL WJORAL ~~
## 5 wordreading =~ WJORAL WJORAL ~~
## 6 wordreading =~ QRLL WJORAL ~~
## 7 wordreading =~ Barnes WJORAL ~~
## 8 wordreading =~ GORT WJORAL ~~
## 9 wordreading =~ QRIR WJORAL ~~
## 10 wordreading =~ WJORAL WJORAL ~~
## 11 wordreading =~ Barnes WJORAL ~~
## 12 wordreading =~ GORT WJORAL ~~
## 13 wordreading =~ QRIR WJORAL ~~
## 14 wordreading =~ QRIR WJORAL ~~
## 15 wordreading =~ WJORAL WJORAL ~~
## 16 wordreading =~ WJORAL WJORAL ~~
## 17 comp =~ PIATR WJORAL ~~
## 18 comp =~ PIATS WJORAL ~~
## 19 wordreading =~ WJORAL WJORAL ~~
## 20 comp =~ PIATS WJORAL ~~
## 21 comp =~ PIATS WJORAL ~~
## 22 comp =~ WordR WJORAL ~~
## 23 wordreading =~ WJORAL WJORAL ~~
## 24 wordreading =~ QRLL QRLL ~~
## 25 wordreading =~ QRLL QRLL ~~
## 26 wordreading =~ QRLL QRLL ~~
## 27 wordreading =~ QRIR Barnes ~~
## 28 wordreading =~ Barnes Barnes ~~
## 29 comp =~ PIATR Barnes ~~
## 30 comp =~ PIATS Barnes ~~
## 31 wordreading =~ Barnes Barnes ~~
## 32 comp =~ PIATS Barnes ~~
## 33 comp =~ PIATS Barnes ~~
## 34 comp =~ WordR Barnes ~~
## 35 wordreading =~ Barnes Barnes ~~
## 36 wordreading =~ GORT GORT ~~
## 37 comp =~ PIATS GORT ~~
## 38 comp =~ WordR GORT ~~
## 39 wordreading =~ QRIR QRIR ~~
## 40 comp =~ PIATS QRIR ~~
## 41 comp =~ WordR QRIR ~~
## 42 wordreading =~ QRIR QRIR ~~
## 43 comp =~ PIATR PIATC ~~
## 44 comp =~ PIATS PIATC ~~
## 45 comp =~ WordR PIATC ~~
## 46 comp =~ PIATR PIATR ~~
## 47 comp =~ PIATS PIATR ~~
## 48 comp =~ WordR PIATR ~~
## 49 comp =~ PIATR PIATR ~~
## 50 comp =~ PIATS PIATR ~~
## 51 comp =~ WordR PIATR ~~
## 52 PIATC ~~ PIATS PIATR ~~
## 53 comp =~ PIATS PIATS ~~
## 54 comp =~ WordR PIATS ~~
## 55 comp =~ PIATR WJORAL ~~
## 56 comp =~ WordR WJORAL ~~
## 57 comp =~ WordR PIATR ~~
## 58 comp =~ PIATR PIATR ~~
## 59 comp =~ PIATS PIATR ~~
## 60 comp =~ WordR PIATR ~~
## 61 PIATC ~~ PIATS PIATR ~~
## 62 comp =~ PIATR PIATS ~~
## resulting_rhs resulting_sepc direction
## 1 WJORAL 0.4402118 positive
## 2 Barnes 0.4079152 positive
## 3 QRIR 0.4019959 positive
## 4 QRLL -0.1437004 positive
## 5 Barnes -0.1812322 positive
## 6 Barnes 0.1225545 positive
## 7 Barnes -0.1276002 positive
## 8 Barnes 0.1143561 positive
## 9 Barnes 0.1347085 positive
## 10 GORT -0.1702306 positive
## 11 GORT 0.1044574 positive
## 12 GORT -0.1156439 positive
## 13 GORT 0.1066411 positive
## 14 QRIR -0.1503392 positive
## 15 WJPC -0.5426127 positive
## 16 PIATC -0.2095393 positive
## 17 PIATR 0.1626955 positive
## 18 PIATR -0.1507294 positive
## 19 PIATR 0.2484495 positive
## 20 PIATS 0.4599847 positive
## 21 WordR -0.1380107 positive
## 22 WordR 0.3848834 positive
## 23 WordR 0.1938631 positive
## 24 Barnes -0.1157583 positive
## 25 PIATR 0.1350430 positive
## 26 WordR 0.1070268 positive
## 27 QRIR -0.1232793 positive
## 28 WJPC -0.1799263 positive
## 29 PIATR 0.1227241 positive
## 30 PIATR -0.1157491 positive
## 31 PIATR 0.1434797 positive
## 32 PIATS 0.3463809 positive
## 33 WordR -0.1059922 positive
## 34 WordR 0.2902511 positive
## 35 WordR 0.1128182 positive
## 36 PIATR 0.1247824 positive
## 37 PIATS 0.2760557 positive
## 38 WordR 0.2315530 positive
## 39 PIATR 0.1413604 positive
## 40 PIATS 0.1291949 positive
## 41 WordR 0.1084263 positive
## 42 WordR 0.1117776 positive
## 43 PIATR -0.1235202 positive
## 44 PIATS -0.1440976 positive
## 45 WordR -0.2927169 positive
## 46 PIATS -0.2068427 positive
## 47 PIATS -0.8201808 positive
## 48 PIATS 0.1211448 positive
## 49 WordR -0.1690451 positive
## 50 WordR 0.4035614 positive
## 51 WordR -0.4080407 positive
## 52 WordR 0.1115527 positive
## 53 WordR -0.8297685 positive
## 54 WordR -0.5309410 positive
## 55 PIATR -0.1053887 negative
## 56 WordR -0.1040664 negative
## 57 PIATS -0.3268497 negative
## 58 WordR 0.1340139 negative
## 59 WordR -0.5060939 negative
## 60 WordR 0.1559392 negative
## 61 WordR -0.1129204 negative
## 62 WordR -0.2848360 negativeFor the baseline model, the standardized solution defines a valid population model, allowing the compensatory-effect check to be conducted. The results indicate that some perturbed parameters produce EPCs exceeding the SESOI. Consequently, the compensatory effect is pronounced for this model under the current SESOI.
Importantly, a “pronounced” result does not imply that EPC equivalence testing is invalid. Rather, it indicates that even trivial misspecifications—approximately 75% of the SESOI—can generate EPCs exceeding the SESOI. Under such conditions, achieving a global trivial-misspecification decision is unlikely unless the true misspecifications are considerably smaller than the SESOI. Substantial misspecifications remain detectable, but their magnitudes may be inflated.
Thus, a pronounced result signals that trivial equivalence is difficult to attain under the current model specification and SESOI.
Suppose that the residual covariance between WJPC and
WordR is theoretically defensible, reflecting shared method
variance or overlapping item content. This parameter is therefore freed
in the model.
model_step1 <- '
comp =~ WJORAL + QRLL + Barnes + GORT + QRIR + WJPC + PIATC
wordreading =~ WJPC + PIATC + PIATR + PIATS + WordR
QRLL ~~ QRIR
WJPC ~~ WordR
'
fit_step1 <- cfa(model_step1, sample.cov = cor_mat, sample.nobs = N)
epc_step1 <- epcEquivFit(fit_step1, stdLoad = 0.4, cor = 0.1)
summary(epc_step1)## Global EPC Evaluation Summary
## --------------------------------
##
## [1. EPC Equivalence Testing: CI-based]
## Substantially Misspecified (M): 4
## Inconclusive (I): 13
## CI-Underpowered (U): 0
## Trivial / Not Misspecified (NM): 34
##
## Global EPC Equivalence Decision: SUBSTANTIAL MISSPECIFICATION
##
## 1.1 Top Substantially Misspecified EPCs (ranked by |std.epc / SESOI|):
## lhs op rhs std.epc std.sesoi severity
## 77 PIATR ~~ WordR 0.291 0.1 2.909
## 76 PIATR ~~ PIATS -0.208 0.1 2.083
## 57 Barnes ~~ PIATS -0.187 0.1 1.871
## 59 GORT ~~ QRIR -0.158 0.1 1.576
##
## 1.2 Top CI-inconclusive EPCs
## (ranked by distance to equivalence bounds; larger = needs narrower CI):
## lhs op rhs lower.std.epc upper.std.epc ci_gap
## 71 WJPC ~~ PIATR -0.144 -0.042 0.044
## 37 WJORAL ~~ Barnes 0.060 0.134 0.034
## 74 PIATC ~~ PIATS 0.078 0.140 0.022
## 72 WJPC ~~ PIATS 0.059 0.119 0.019
## 73 PIATC ~~ PIATR 0.020 0.118 0.018After freeing this dominant residual covariance, four substantially misspecified fixed parameters remain, and the global EPC equivalence decision is still substantial misspecification.
epcEquivCheck(fit_step1)## EPC Equivalence Compensatory Effect (Standardized Parameters)
## ----------------------------------------------------
## Feasible standardized population: TRUE
## No EPC exceeded the SESOI under tested perturbations.However, the compensatory-effect check now yields a not pronounced result. Perturbations up to 75% of the SESOI no longer generate EPCs exceeding the SESOI. This indicates that, under the revised model, a trivial-misspecification decision is feasible in principle.
This contrast highlights an important point: a pronounced result does not imply that EPC equivalence testing is uninformative. Rather, it may indicate that a dominant misspecification is distorting the EPCs of other fixed parameters (and potentially inflating its own EPC as well). Once that dominant parameter is freed, compensatory distortions may be reduced, and classifications of substantial EPCs become more reliable indicators of genuine misspecification.
Next, we free the residual covariance between PIATR and
WordR, assuming theoretical justification. This adjustment
reduces the number of substantially misspecified parameters from four to
two, but the global decision remains substantial
misspecification.
model_step2 <- '
comp =~ WJORAL + QRLL + Barnes + GORT + QRIR + WJPC + PIATC
wordreading =~ WJPC + PIATC + PIATR + PIATS + WordR
QRLL ~~ QRIR
WJPC ~~ WordR
PIATR ~~ WordR
'
fit_step2 <- cfa(model_step2, sample.cov = cor_mat, sample.nobs = N)
epc_step2 <- epcEquivFit(fit_step2, stdLoad = 0.4, cor = 0.1)
summary(epc_step2)## Global EPC Evaluation Summary
## --------------------------------
##
## [1. EPC Equivalence Testing: CI-based]
## Substantially Misspecified (M): 2
## Inconclusive (I): 11
## CI-Underpowered (U): 0
## Trivial / Not Misspecified (NM): 37
##
## Global EPC Equivalence Decision: SUBSTANTIAL MISSPECIFICATION
##
## 1.1 Top Substantially Misspecified EPCs (ranked by |std.epc / SESOI|):
## lhs op rhs std.epc std.sesoi severity
## 58 Barnes ~~ PIATS -0.194 0.1 1.940
## 60 GORT ~~ QRIR -0.157 0.1 1.575
##
## 1.2 Top CI-inconclusive EPCs
## (ranked by distance to equivalence bounds; larger = needs narrower CI):
## lhs op rhs lower.std.epc upper.std.epc ci_gap
## 74 PIATC ~~ PIATR 0.058 0.146 0.042
## 38 WJORAL ~~ Barnes 0.053 0.128 0.028
## 64 GORT ~~ PIATS 0.065 0.125 0.025
## 76 PIATC ~~ WordR -0.114 -0.044 0.014
## 75 PIATC ~~ PIATS 0.029 0.112 0.012epcEquivCheck(fit_step2)## EPC Equivalence Compensatory Effect (Standardized Parameters)
## ----------------------------------------------------
## Feasible standardized population: FALSE
## Any EPC exceeding SESOI: NA
## Compensatory Effect: NOT APPLICABLEAt this stage, epcEquivCheck() is not applicable because
the standardized parameter estimates no longer define a valid population
correlation matrix. This does not invalidate EPC equivalence testing
itself; it merely means that the compensatory-effect diagnostic cannot
be computed under these conditions.
In Steps 3–5, additional measurment error covariances are freed
(Barnes ~~ PIATS, GORT ~~ QRIR, and
WJORAL ~~ QRIR) to further illustrate EPC-based
diagnostics. Each step reduces the number of substantially misspecified
EPCs, eventually eliminating them entirely.
model_step3 <- '
comp =~ WJORAL + QRLL + Barnes + GORT + QRIR + WJPC + PIATC
wordreading =~ WJPC + PIATC + PIATR + PIATS + WordR
QRLL ~~ QRIR
WJPC ~~ WordR
PIATR ~~ WordR
Barnes ~~ PIATS
'
fit_step3 <- cfa(model_step3, sample.cov=cor_mat, sample.nobs=N)
epc_step3 <- epcEquivFit(fit_step3, stdLoad = 0.4, cor = 0.1)
summary(epc_step3)## Global EPC Evaluation Summary
## --------------------------------
##
## [1. EPC Equivalence Testing: CI-based]
## Substantially Misspecified (M): 1
## Inconclusive (I): 12
## CI-Underpowered (U): 0
## Trivial / Not Misspecified (NM): 36
##
## Global EPC Equivalence Decision: SUBSTANTIAL MISSPECIFICATION
##
## 1.1 Top Substantially Misspecified EPCs (ranked by |std.epc / SESOI|):
## lhs op rhs std.epc std.sesoi severity
## 60 GORT ~~ QRIR -0.161 0.1 1.606
##
## 1.2 Top CI-inconclusive EPCs
## (ranked by distance to equivalence bounds; larger = needs narrower CI):
## lhs op rhs lower.std.epc upper.std.epc ci_gap
## 74 PIATC ~~ PIATR 0.064 0.151 0.036
## 72 WJPC ~~ PIATR 0.024 0.132 0.032
## 39 WJORAL ~~ Barnes 0.049 0.122 0.022
## 77 PIATR ~~ PIATS -0.193 -0.090 0.010
## 78 PIATS ~~ WordR 0.030 0.110 0.010epcEquivCheck(fit_step3)## EPC Equivalence Compensatory Effect (Standardized Parameters)
## ----------------------------------------------------
## Feasible standardized population: FALSE
## Any EPC exceeding SESOI: NA
## Compensatory Effect: NOT APPLICABLEmodel_step4 <- '
comp =~ WJORAL + QRLL + Barnes + GORT + QRIR + WJPC + PIATC
wordreading =~ WJPC + PIATC + PIATR + PIATS + WordR
QRLL ~~ QRIR
WJPC ~~ WordR
PIATR ~~ WordR
Barnes ~~ PIATS
GORT ~~ QRIR
'
fit_step4 <- cfa(model_step4, sample.cov=cor_mat, sample.nobs=N)
epc_step4 <- epcEquivFit(fit_step4, stdLoad = 0.4, cor = 0.1)
summary(epc_step4)## Global EPC Evaluation Summary
## --------------------------------
##
## [1. EPC Equivalence Testing: CI-based]
## Substantially Misspecified (M): 1
## Inconclusive (I): 10
## CI-Underpowered (U): 0
## Trivial / Not Misspecified (NM): 37
##
## Global EPC Equivalence Decision: SUBSTANTIAL MISSPECIFICATION
##
## 1.1 Top Substantially Misspecified EPCs (ranked by |std.epc / SESOI|):
## lhs op rhs std.epc std.sesoi severity
## 42 WJORAL ~~ QRIR -0.208 0.1 2.078
##
## 1.2 Top CI-inconclusive EPCs
## (ranked by distance to equivalence bounds; larger = needs narrower CI):
## lhs op rhs lower.std.epc upper.std.epc ci_gap
## 74 PIATC ~~ PIATR 0.064 0.151 0.036
## 72 WJPC ~~ PIATR 0.022 0.129 0.029
## 40 WJORAL ~~ Barnes 0.079 0.149 0.021
## 43 WJORAL ~~ WJPC 0.052 0.119 0.019
## 67 QRIR ~~ PIATC 0.082 0.140 0.018epcEquivCheck(fit_step4)## EPC Equivalence Compensatory Effect (Standardized Parameters)
## ----------------------------------------------------
## Feasible standardized population: FALSE
## Any EPC exceeding SESOI: NA
## Compensatory Effect: NOT APPLICABLEmodel_step5 <- '
comp =~ WJORAL + QRLL + Barnes + GORT + QRIR + WJPC + PIATC
wordreading =~ WJPC + PIATC + PIATR + PIATS + WordR
QRLL ~~ QRIR
WJPC ~~ WordR
PIATR ~~ WordR
Barnes ~~ PIATS
GORT ~~ QRIR
WJORAL ~~ QRIR
'
fit_step5 <- cfa(model_step5, sample.cov=cor_mat, sample.nobs=N)
epc_step5 <- epcEquivFit(fit_step5, stdLoad = 0.4, cor = 0.1)
summary(epc_step5)## Global EPC Evaluation Summary
## --------------------------------
##
## [1. EPC Equivalence Testing: CI-based]
## Substantially Misspecified (M): 0
## Inconclusive (I): 11
## CI-Underpowered (U): 0
## Trivial / Not Misspecified (NM): 36
##
## Global EPC Equivalence Decision: INCONCLUSIVE
##
## 1.2 Top CI-inconclusive EPCs
## (ranked by distance to equivalence bounds; larger = needs narrower CI):
## lhs op rhs lower.std.epc upper.std.epc ci_gap
## 74 PIATC ~~ PIATR 0.064 0.149 0.036
## 77 PIATR ~~ PIATS -0.181 -0.080 0.020
## 41 WJORAL ~~ Barnes 0.044 0.120 0.020
## 40 WJORAL ~~ QRLL -0.155 -0.082 0.018
## 78 PIATS ~~ WordR 0.033 0.111 0.011epcEquivCheck(fit_step5)## EPC Equivalence Compensatory Effect (Standardized Parameters)
## ----------------------------------------------------
## Feasible standardized population: FALSE
## Any EPC exceeding SESOI: NA
## Compensatory Effect: NOT APPLICABLEAfter Step 5, no fixed parameters are classified as substantially misspecified. Eleven parameters remain inconclusive, and 36 are trivially misspecified, yielding a global decision of inconclusive.
At this point, EPC equivalence testing suggests that the model is close to trivial misspecification but lacks sufficient precision to reach a definitive conclusion under the current SESOI.
Given an inconclusive global decision, researchers have two primary options: (a) increase sample size to obtain narrower EPC confidence intervals, or (b) relax the SESOI to reflect a more tolerant definition of substantive equivalence.
For illustration, we increase the SESOI for measurement error correlations to .19. Under this relaxed criterion, all fixed parameters are classified as trivially misspecified, yielding a global decision of equivalent.
epc_step5_newsesoi <- epcEquivFit(fit_step5, stdLoad = 0.4, cor = 0.19)
summary(epc_step5_newsesoi)## Global EPC Evaluation Summary
## --------------------------------
##
## [1. EPC Equivalence Testing: CI-based]
## Substantially Misspecified (M): 0
## Inconclusive (I): 0
## CI-Underpowered (U): 0
## Trivial / Not Misspecified (NM): 47
##
## Global EPC Equivalence Decision: EQUIVALENTSubstantively, this result indicates that the model can be regarded as approximately correct if researchers are willing to tolerate residual correlations of this magnitude. Otherwise, additional data would be required to reach a definitive conclusion under the original SESOI.
This appendix demonstrates how epcEquivFit() can be used
to implement EPC-based equivalence testing for both global and local
model evaluation. We illustrated how EPC-based diagnostics evolve as
model specification changes and how conclusions depend on the chosen
SESOI.
We also demonstrated the role of epcEquivCheck(), which
assesses whether compensatory effects may inflate EPCs such that trivial
misspecifications produce apparently substantial EPCs. When this occurs,
trivial equivalence becomes difficult to achieve, and substantial EPCs
must be interpreted with caution. Analysts should evaluate whether
theoretically justified model adjustments address the underlying
misspecification rather than merely compensatory distortions.
Finally, we emphasize that EPC-based equivalence testing is not intended to legitimize post-hoc model modification. Rather, it provides a principled framework for evaluating whether a model can be treated as approximately correct relative to substantively meaningful effect sizes. Substantive theory and cross-validation remain essential.