In Table 30.1 - the hypothesis is "this item has the same difficulty for two groups"
In Table 30.2, 30.3 - the hypothesis is "this item has the same difficulty as its average difficulty for all groups" - this is the best estimate when the data fit the model
In Table 30.4 - the hypothesis is "this item has no overall DIF across all groups"
Table 30.4 summarizes the CLASS/GROUP Differential Item Functioning statistics for each item shown in Table 30.2. Table 30.2 shows a t-statistic for each CLASS/GROUP. These are summarized as chi-square statistics for each item, indicating whether the observed DIF within each item is due to chance alone. The null hypothesis is that the DIF is statistically zero across the classes. Please do not use anchoring (IAFILE=, PAFILE=, SAFILE=) when investigating this Table.
DIF class/group specification is: DIF=@GENDER
----------------------------------------------------------------------------------
| KID SUMMARY DIF BETWEEN-CLASS TAP |
| CLASSES CHI-SQUARE D.F. PROB. UNWTD MNSQ t=ZSTD Number Name |
|--------------------------------------------------------------------------------|
| 2 .8772 1 .3490 .9200 .4132 6 3-4-1 |
| 2 2.2427 1 .1342 2.5792 1.2592 7 1-4-3-2 |
| 2 1.9207 1 .1658 1.9963 1.0211 13 1-4-3-2-4 |
----------------------------------------------------------------------------------
KID CLASSES/GROUPS is the count of person CLASSES or GROUPS for the item: 2 = Male and Female.
SUMMARY DIF CHI-SQUARE is the sum of the t-statistic values from Table 30.2, squared and normalized. See Example 1 below.
D.F. is the degrees of freedom, the count of CLASSES or GROUPS contributing to the chi-square less 1.
PROB. is the probability of the chi-square. Values less than .05 indicate statistical significance.
BETWEEN-CLASS are Between-Group Fit Statistics, testing the hypothesis: "The dispersion of the group measures accords with Rasch model expectations."
UNWTD MNSQ is the unweighted mean-square (chi-square divided by its degrees of freedom). It is the size of the misfit (expectation = 1.0, overfit <1.0, underfit >1.0).
t=ZSTD is the significance of the MEAN-SQUARE standardized as a unit-normal deviate (t-statistic with infinite degrees of freedom).
TAP is the item. Number is the item entry number. Name is the item label.
Item-Trait Interaction Chi-Square - RUMM2020 Item-Trait Chi-Square and Winsteps DIF Size
The SUMMARY DIF CHI-SQUARE values in Table 30.4 are equivalent to the item-trait chi-square statistics reported by RUMM with DIF=MA3 (or however many strata are chosen in RUMM). In RUMM, the trait CLASSes are obtained by ordering the persons by measure, omitting extreme scores, and dividing the ordered list as equally as possible into equal size classes, including all persons with the same measure in the same class.
The Root Mean Square Error of Approximation (RMSEA)
RMSEA is a transformation of the Item-Trait Chi-Square and so of the SUMMARY DIF CHI-SQUARE with DIF=MA3. Its computation is
RMSEA = √ max( [((χ²/df) - 1)/(N - 1)] , 0)
where
χ² = SUMMARY DIF CHI-SQUARE
d.f. = D.F.
N is the person sample size
Example 1. SUMMARY DIF CHI-SQUARE.
Let's do an analysis of Exam1.txt with Udecimals=4. Output Table 30 with DIF=@Gender
In Table 30.2, the values for item 13:
CLASS F: COUNT= 18, DIF SIZE= -.7268, DIF S.E.= .6764
CLASS M: COUNT= 16, DIF SIZE= -.7075, DIF S.E.= .7497
So the t-statistics are approximately:
CLASS F: t = -.7268/.6764 with (18-1) d.f. -> -1.0745 with 17 d.f.
CLASS M: t = -.7075/.7497 with (16-1) d.f. -> .9437 with 15 d.f.
Squared and normalized. The normalizing transformation is Peizer and Pratt (1968) in the Handbook of the Normal Distribution, Second Edition (Patel and Read), p. 214.
CLASS F: t**2 = 1.1546 -> normalized(t**2) = 1.0843
CLASS M: t**2 = .8905 -> normalized(t**2) = .8363
Chi-square = 1.0843 + .8363 = 1.9207 with (2-1) = 1 d.f.
Andersen’s Likelihood-Ratio Test (LRT, 1973) is a chi-squared test that the item difficulties are statistically the same for different subgroups of persons, so this a generalized DIF (Differential Item Functioning) test. A typical grouping is by raw score, for instance a high-low spit of the person sample. If the sample is large enough, each raw score could be a person subgroup. Winsteps does not compute LRT, but an equivalent chi-squared computation is in Winsteps 30.4. LRT lumps all items together, but Table 30.4 reports each item separately. So LRT is equivalent to summing the CHI-SQUARE and D.F. columns in 30.4 and looking up that chi-squared probability. If you wish to do a high-low score split, then DIF=MA2. .
If you wish to compute LRT, then the easiest way is to use Winsteps Output Files IPMATRIX= (omit extreme scores, omit person identifiers) to output the dichotomous scored responses to R Statistics. And then analyze the data with R package eRm and output LRtest. For instance:
> install.packages("eRm")
> library(eRm)
> res <- RM(data)
> LRtest(res, splitcr = "median", se = TRUE)
