Table 5 reports summary statistics about the data for the analysis.
+--------------------------------------------------+
| Cat Score Exp. Resd StRes| |
|-----------------------------+--------------------|
| 4.80 4.80 4.80 .00 .00 | Mean (Count: 1152) |
| 1.63 1.63 1.03 1.27 .99 | S.D. (Population) |
| 1.64 1.64 1.03 1.27 .99 | S.D. (Sample) |
+--------------------------------------------------+
Column headings have the following meanings:
Cat = Observed value of the category as entered in the data file.
Score = Value of category after it has been recounted cardinally commencing with "0" corresponding to the lowest observed category.
Exp. = Expected score based on current estimates
Resd = Residual, the score difference between Step and Exp.
StRes = The residual standardized by its standard error. StRes is expected to approximate a unit normal distribution.
Mean = average of the observations
Count = number of observations
S.D. (Population) = standard deviation treating this sample as the entire population
S.D. (Sample) = standard deviation treating this sample as a sample from the population. It is larger than S.D. (Population).
The raw-score error variance % is 100*(Resd S.D./Cat S.D.)²
When the parameters are successfully estimated, the mean Resd is 0.0. If not, then there are estimation problems - usually due to too few iterations, or anchoring.
When the data fit the Rasch model, the mean of the "StRes" (Standardized Residuals) is expected to be near 0.0, and the "S.D." (sample standard deviation) is expected to be near 1.0. These depend on the distribution of the residuals.
Explained variance by each facet can be approximated by using the element S.D.^2 (^2 means "squared").
From Table 5:
Explained variance = Score Population S.D.^2 - Resd^2
Explained variance % = Explained variance * 100 / Score Population S.D.^2
From Table 7:
|-------------------------------+--------------+---------------------+------+-------------+ +---------------------|
| 460.8 96.0 4.8 4.73| .00 .08 | 1.00 -.1 .99 -.2| | .61 | | Mean (Cnt: 12) |
| 29.5 .0 .3 .32| .19 .00 | .23 1.8 .22 1.7| | .05 | | S.D. (Population) |
| 30.8 .0 .3 .33| .20 .00 | .24 1.9 .23 1.8| | .06 | | S.D. (Sample) |
+------------------------------------------------------------------------------------------ ----------------------+
v1 = (measure Population S.D. facet 1)^2
v2 = (measure Population S.D. facet 2)^2
v3 = (measure Population S.D. facet 3)^2
vsum = v1 + v2 + v3 + .... (for all facets)
Compute Explained variance for each facet:
Explained variance % by facet 1 = (Explained variance %) * v1 /vsum
Explained variance % by facet 2 = (Explained variance %) * v2 /vsum
Explained variance % by facet 3 = (Explained variance %) * v3 /vsum
Example: Guilford.txt - a 3 facet analysis:
In Table 5:
Raw-score variance of observations = 3.526 100.00%
Variance explained by Rasch measures = 1.446 41.02%
Variance of residuals = 2.080 58.98%
In Table 7:
measure Population S.D. facet 1 = 0.13, variance = 0.02, % total variance = (0.02/0.29) * 41.02 = 2.83%
measure Population S.D. facet 2 = 0.42, variance = 0.18, % total variance = (0.18/0.29) * 41.02 = 25.47%
measure Population S.D. facet 3 = 0.30, variance = 0.09, % total variance = (0.09/0.29) * 41.02 = 12.73%
variance sum = 0.29
Confirmation of variance explained: make all facets Positive=1,2,3 so that everything is addition. Then we want the S.D. of the sums of the 3 element measures. Easy! Output the Residualfile= to Excel and obtain the S.D. of the Meas column. We can then use www.rasch.org/rmt/rmt221j.htm with this S.D. as, say, the "person" S.D., and the item S.D. as zero.
For a Rasch-based Generalizability Coefficient:
G = (Explained variance% by target facet) / 100
A more specific Generalizability Coefficient can be formulated by selecting appropriate variance terms from Table 5, Table 7, and Table 13.
Estimable observations = 1152, Free parameters = 21, Degrees of freedom = 1129
Global Pearson chi-squared = 1137.55, probability = .4233
Count Mean S.D.
Responses after end-of-file = 0 0.00 0.00
Responses only in extreme scores = 0 0.00 0.00
Responses in two extreme scores = 0 0.00 0.00
Responses with invalid elements = 0 0.00 0.00
Responses invalid after recounting = 0 0.00 0.00
Responses non-extreme estimable = 1152 4.80 1.63
Responses in one extreme score = 0 0.00 0.00
All Responses = 1152 4.80 1.63
Identification
|
Meaning
|
Estimable observations
|
(Weighted) count of all observations in non-extreme (minimum possible or maximum possible) response strings
|
Free parameters
|
Count of the minimum number of elements and thresholds which, when estimated, decide the estimates of all other elements.and thresholds
|
Degrees of freedom
|
The d.f. are the estimable observations less the free parameters
|
Global Pearson chi-squared
|
This summarizes the fit of all the data to the Rasch model. It is the sum of the squared standardized residuals (weighted if weights are specified). It assumes that the element measures are the maximum-likelihood estimates.
To produce this value from the Residual File:
Output Files menu
Residual/Response File
Select fields to output
Decimal places 4
OK
Output to Excel
Insert blank column
Divide residual^2 / variance into the blank column
Sum the blank column (for more accuracy, only for Status 1 observations)
Note: there will be small rounding errors.
|
Probability
|
The probability of observing the chi-squared value (or larger) when the data fit the model
|
|
|
Response Type
|
Responses not used for estimation: see Residual File
|
Responses after end-of-file
|
A Facets internal work-file has too many responses. Please report this to Winsteps.com and rerun this analysis.
|
Responses only in extreme scores
|
The category of the rating scale cannot be estimated.
|
Responses in two extreme scores
|
These cannot be estimated nor used for estimating element measures.
|
Responses with invalid elements
|
Elements for these observations are not defined. See Table 2 with Build option.
|
Responses invalid after recounting
|
A dichotomy or rating scale has less than two categories, so it cannot be estimated. See Table 8 for missing or one-category rating scales.
|
Response Type
|
Responses used for estimation: see Residual File
|
Responses non-extreme estimable
|
This is the count of responses used in estimating non-extreme parameter values (element measures and rating scale structures).
|
Responses in one extreme score
|
These are only used for estimating the element with the extreme score
|
All Responses
|
Shown if there is more than one response type listed above
|
Count of measurable responses = 1152
Raw-score variance of observations = 2.67 100.00%
Variance explained by Rasch measures = 1.06 39.57%
Variance of residuals = 1.61 60.43%
Variance explained by bias/interactions = 0.14 5.24%
Variance remaining in residuals = 1.47 55.06%
An approximate Analysis of Variance (ANOVA) of the data
|
Identification
|
Meaning
|
Count of measurable responses
|
All responses (including for extreme scores) with weighting (if any)
|
Raw-score variance of observations
|
Square of S.D. (Population) of Score
|
Variance explained by Rasch measures
|
Raw score variance - Variance of residuals. This is dominated by the spread of the elements. We usually want the raters to be equally lenient = explain no variance, etc.
The size of the expected variance for 2-facet models is shown in www.rasch.org/rmt/rmt221j.htm
|
Variance of residuals
|
Square of S.D. (Population) of Resd.
|
Variance explained by bias/interactions
|
The variance explained by the bias/interactions specified with "B" in your Models= statements
|
Variance remaining in residuals
|
Variance of residuals - Variance of interactions
|
Nested models: Suppose we want to estimate the effect on fit of a facet.
Run twice:
First analysis: 3 facets
Models = ?,?,?, R
Second analysis: 2 facets:
Models = ?,?,X,R
We can obtain an estimate of the improvement of fit based on including the third facet:
Chi-squared of improvement = chi-squared (2 facets) - chi-squared (3 facets) with d.f. (count of elements in facet 3 - 1).
If global fit statistics are the decisive evidence for choice of analytical model, then Facets is not suitable. In the statistical philosophy underlying Facets, the decisive evidence for choice of model is "which set of measures is more useful" (a practical decision), not "which set of measures fit the model better" (a statistical decision). The global fit statistics obtained by analyzing your data with log-linear models (e.g., in SPSS) will be more exact than those produced by Facets.
