Please do not interpret this as a usual factor analysis. These plots show contrasts between opposing factors, identified as "A,B,.." and "a,b,...", not loadings on one factor. For more discussion, see dimensionality and contrasts.
Quick summary:
(a) the X-axis is the measurement axis. So we are not concerned about quadrants, we are concerned about vertical differences. The Table 23 plots show contrasts between types of items: those at the top vs. those at the bottom. The Table 24 plots show contrasts between types of persons: those at the top vs. those at the bottom.
(b) "How much" is important. See the Variance Table explained in Table 23.0. Important differences have eigenvalues greater than 2.0.
(c) If the difference is important, it suggests that we divide the test into pieces, clustering the items in the top half of the plot and the items in the bottom half. Winsteps estimates a measure for each person on each cluster of items. The correlations of the measures are reported. Small disattenuated correlations indicate that the clusters of items measure different sub-dimensions.
(d) Perform separate analyses for the target clusters of items. Cross-plot and correlate the person measures. We will then see for whom the differences are important. Usually, for a carefully designed instrument, it is such a small segment that we decide it is not worth thinking of the test as measuring two dimensions. Tables 23.4 also helps us think about this.
1. Put a code into the item label to indicate the item subset to which the item belongs.
2a. Use ISELECT= for each subset code, and produce a person measure (PFILE=). Cross-plot the person measures.
or
2b. Do a Differential Person Functioning (DPF=) analysis based on the subset code. Table 31.1 will give you an inter-item-subset t-test for each person.
Table 23.0 Variance components scree plot for items
Table 23.1, 23.11 Principal components plots of item loadings
Table 23.2, 23.12 Item Principal components analysis/contrast of residuals
Table 23.3, 23.13 Item contrast by persons
Table 23.4, 23.14 Item contrast loadings sorted by measure
Table 23.5, 23.15 Item contrast loadings sorted by entry number
Table 23.6, 23.16 Person measures for item clusters in contrast. Cluster Measure Plot for Table 23.6.
Table 23.99 Largest residual correlations for items
Youtube video explaining Table 23
These plots show the contrasts by plotting the unstandardized "raw" loading on each component against the item calibration (or person measure). The contrast shows items (or persons) with different residual patterns. A random pattern with few high loadings is expected.
The horizontal axis is the Rasch dimension. This has been extracted from the data prior to the analysis of residuals.
Letters "A,B,C,..." and "a,b,c,..." identify items (persons) with the most opposed loadings on the first contrast in the residuals. On subsequent contrasts, the items retain their first contrast identifying letters. When there are 9 items (persons) or less, the item number is displayed.
The items are clustered into 3 clusters on the right-side of the plot. This is because interpreting the PCA Components ("Contrasts") usually requires us to compare the items at the top of the plot against the items at the bottom, often ignoring the middle items. The purpose of the plot associated with Table 23.6 is to help us see whether the 3 clusters of items are truly measuring different things. If they are measuring the same thing statistically, then usually no action is needed. If the clusters of items are measuring different things, then the analyst must decide what to do.
In the residuals, each item (person) is modeled to contribute one unit of randomness. Thus, there are as many residual variance units as there are items (or persons). For comparison, the amount of person (item) variance explained by the item (person) measures is approximated as units of that same size.
In the Figure below from Example0.txt , the first contrast in the standardized residuals separates the items into 3 clusters. To identify the items, see Tables 23.3, 24.3. In this example, the dimension is noticeable, with strength of around 5 out of 25 items. This is in the residual variance, i.e., in the part of the observations unexplained by the measurement model. But, hopefully, most of the variance in the observations has been explained by the model. The part of that explained variance attributable to the Persons is shown in variance units locally-rescaled to accord with the residual variances. In this example, the variance explained by the person measures is equivalent to 10 items. Consequently, the secondary dimension in the items is noticeable. The disattenuated correlation between person measures on item in Cluster 1 and person measures on items in Cluster 3 is less than 0.3. The secondary dimension underlying the 1st Contrast is biasing the person measures.
Letters display when there are more than 9 items on the plot. Numbers display when there are 9 or less items on the plot.
For items:
Table of STANDARDIZED RESIDUAL variance in Eigenvalue units = ACT information units
Eigenvalue Observed Expected
Total raw variance in observations = 50.9521 100.0% 100.0%
Raw variance explained by measures = 25.9521 50.9% 50.7%
Raw variance explained by persons = 10.3167 20.2% 20.2%
Raw Variance explained by items = 15.6354 30.7% 30.6%
Raw unexplained variance (total) = 25.0000 49.1% 100.0% 49.3%
Unexplned variance in 1st contrast = 4.6287 9.1% 18.5%
STANDARDIZED RESIDUAL CONTRAST 1 PLOT
-4 -3 -2 -1 0 1 2 3 4 5
-+------+------+------+------+------+------+------+------+------+- COUNT CLUSTER
.8 + | +
| | A B | 2 1
.7 + | +
| | |
.6 + | C + 1 1
| | |
C .5 + | +
O | | |
N .4 + | DE + 2 1
T | | F | 1 1
R .3 + | +
A | | |
S .2 + | G + 1 2
T | | |
.1 + | +
1 | | H | 1 2
.0 +----------------------------|-----------------------------------+
L | | I | 1 2
O -.1 + K | J + 2 2
A | |L | 1 2
D -.2 + | +
I | |M l | 2 3
N -.3 + j | k + 2 3
G | i | | 1 3
-.4 + f h g | + 3 3
| | |
-.5 + e | + 1 3
| d c | | 2 3
-.6 + b | + 1 3
| a | | 1 3
-+------+------+------+------+------+------+------+------+------+-
-4 -3 -2 -1 0 1 2 3 4 5
ACT MEASURE
COUNT: 1 1 1 11 111 21 21221 1 21 1 1
KID 1 2 112 1 5346832333342 23222 1 1 1 1 1 11
T S M S T
%TILE 0 10 20 40 50 60 70 80 90 99
If your plot looks like this, please change the font in your text editor (NotePad) to "fixed width": Courier New, Consolas, Lucida Console
-4 -3 -2 -1 0 1 2 3 4 5
-+------+------+------+------+------+------+------+------+------+- COUNT CLUSTER
.8 + | +
| | A B | 2 1
.7 + | +
| | |
.6 + | C + 1 1
| | |
C .5 + | +
O | | |
N .4 + | DE + 2 1
T | | F | 1 1
R .3 + | +
A | | |
S .2 + | G + 1 2
T | | |
.1 + | +
1 | | H | 1 2
.0 +----------------------------|-----------------------------------+
L | | I | 1 2
O -.1 + K | J + 2 2
A | |L | 1 2
D -.2 + | +
I | |M l | 2 3
N -.3 + j | k + 2 3
G | i | | 1 3
-.4 + f h g | + 3 3
| | |
-.5 + e | + 1 3
| d c | | 2 3
-.6 + b | + 1 3
| a | | 1 3
-+------+------+------+------+------+------+------+------+------+-
-4 -3 -2 -1 0 1 2 3 4 5
Example: A teacher survey had hundreds of items contributed by numerous stakeholders (with special agendas), but almost no hypothesized constructs. In other words, it was a mess! PCA of residuals was used to discover subsets of items that cooperated. A useful strategy is to use IWEIGHT=. Weight a core subset of items on a construct of interest "1", and weight all the other items "0". Then do a Winsteps analysis. All the items will be calibrated, but only the weighted items will contribute to the person measures. Then do a PCA of residuals. All the items will participate. The core items will cluster. Unweighted items that cluster with the core can be inspected, and added to the core if suitable. Then the process is repeated. When all the items for one construct have been identified, those items can be deleted from the dataset. The process begins again with the next core subset of items.
Correlation Table of person Measures on each of the Clusters:
Approximate relationships between the KID measures
PCA ACT Pearson Disattenuated Pearson+Extr Disattenuated+Extr
Contrast Clusters Correlation Correlation Correlation Correlation Cluster Sizes
1 1 - 2 0.2951 0.4683 0.3592 0.5497 6 6
1 1 - 3 0.1411 0.2191 0.1958 0.2941 6 13
1 2 - 3 0.8065 1.0000 0.8123 1.0000 6 13
1 1 - B 0.4541 0.5506 6 25
1 2 - B 0.9030 0.8928 6 25
1 3 - B 0.8934 0.8652 13 25
The items are anchored (fixed) at their difficulties from the main analysis (as reported in Table 14.1). Then the items are segmented into subtests according to their cluster numbers, and each person is measured on each subtest. The person measures for each cluster of items are correlated with their measures from the other clusters of items, and reported here as the "Pearson Correlation". Each person measure for each cluster of items has a standard error. These error variances are removed to produce the "Disattenuated Correlation". If the disattenuated correlation approaches 1.0, then the person measures from the two clusters of items are statistically the same. We cannot reject the hypothesis that the two clusters of items are measuring the same thing. If the disattenuated correlation is out of range or undefined, it is reported as (1.00) or (-1.00) matching the sign of the observed correlation.
The correlation and disattenuated correlation in, say, Table 23.1, are computed from the person measures and model standard errors shown in Table 23.6 . This would be equivalent to using the Model Reliability from Table 3.1. In the computations, the "Real" standard errors would be larger, so the "Real" disattenuated correlations would also be larger. The "Model" disattenuated correlation is a more conservative (lower) estimate, a lower bound. The computation is based on Rasch measures, so Cronbach Alpha, which is based on raw scores, does not apply.
For the Pearson Correlation and the Disattenuated (Pearson) Correlation, persons with extreme scores on a cluster are omitted. For the Pearson+Extr(eme) Correlation and the Disattenuated (Pearson)+Extr(eme) Correlation, persons with extreme scores on a cluster are included. Cluster sizes are the counts of items in each cluster.
These correlations are approximate. For more accurate correlations, please perform separate analyses of each cluster of items, and then use the Scatterplot function to investigate the relationships between the person measures estimated from the different clusters.
The middle cluster (2) usually corresponds roughly with the Rasch dimension. So the important correlations are 1-2 and 2-3. Clusters 1 and 3 are both somewhat off-dimension so the correlation 1-3 is somewhat accidental. It is nice when it is high, but low 1-3 correlation merely tells us that 1 and 3 are off-dimension in different ways.
Roughly speaking, we look at the disattenuated correlations (otherwise measurement error clouds everything):
Correlations below 0.57 indicate that person measures on the two item clusters have half as much variance in common as they have independently. (Cut-off for probably different latent variables?)
Correlations above 0.71 indicate that person measures on the two item clusters have more than half their variance in common, so they are more dependent ( = more the same thing) than independent (= different things).
Correlations above 0.82, twice as dependent as independent. (Cut-off for probably the same latent variable?)
Correlations above 0.87, three times as dependent as independent (definitely the same thing).
Cluster B is the baseline measures, the estimates from the main analysis. No disattenuated correlation is shown for B because B is dependent on the other clusters including the cluster with which it is compared.
Example 1: From Table 23.3 of the analysis of a large empirical dataset:
Approximate relationships between the person measures
Contrast 1 Pearson Disattenuated
Item Clusters Correlation Correlation
1 - 2 .5151 .5762
Actual relationships of person measures estimated from separate analyses of the two clusters of items:
Contrast 1 Pearson Disattenuated
Item Clusters Correlation Correlation
1 - 2 .5083 .6352
