Exact Match: OBS% and EXP%

+---------------------------------------------------------------------------------------------------+

|ENTRY    RAW                   MODEL|   INFIT  |  OUTFIT  |PTMEA|EXACT MATCH|                      |

|NUMBER  SCORE  COUNT  MEASURE  S.E. |MNSQ  ZSTD|MNSQ  ZSTD|CORR.| OBS%  EXP%| KID                  |

|------------------------------------+----------+----------+-----+-----------+----------------------|

|    72     14     25   -1.32     .37|2.02   2.9|5.16   5.7|A .04| 60.0  65.8| JACKSON, SOLOMON     |

 

Suppose your dataset consists of observations, {Xni}, of person n on item i. Based on the Rasch parameters (measures), there is an expected value Eni corresponding to each observation Xni. Eni is obtained by a calculation from the Rasch model.

 

When the absolute value of (Xni-Eni) is less than 0.5 then the observed data point is within 0.5 score points of its expected value, so the match is the closest possible. Thus, across all observations of item i,

Count ( |Xni-Eni| <0.5 ) = A - these observations are of the closest categories to their expectations

Count ( |Xni-Eni| =0.5 ) = B - these observations are on the borderline of matching their expectations

Count ( |Xni-Eni| >0.5 ) = C - these observations are at least one category away from their expectations

 

So that A+B+C = Count (Xni)

 

OBS% = Observed % = 100 * ( A + B/2 ) / ( A+B+C )

 

B/2: consider the situation when person ability = item difficulty with dichotomous observations. Then every observation is 1 or 0, and every expectation is 0.5, so B = 100%. Under these conditions, it would be reasonable to say that 50% (B/2) of the observations matched their expectations.

 

Each possible value of Xni has a probability according to the Rasch model. Based on these, the expected value of OBS% can be computed, this is the EXP%. So, if the possible values of Xni are j=0,1,2,...,m, with probabilities Pnij, then

| j-Eni |<0.5  is either 1 when the condition is true, or zero when it is false, so

A = sum ( ( | j-Eni | < 0.5 )*Pnij ) - this is the sum of all the Pnij for which the absolute value j-Eni is less than 0.5

B = sum ( ( | j-Eni | = 0.5 )*Pnij ) - this is the sum of all the Pnij for which the absolute value j-Eni equals 0.5

C = sum ( ( | j-Eni | > 0.5 )*Pnij ) - this is the sum of all the Pnij for which the absolute value j-Eni is greater than 0.5

 

So that A+B+C = Count (Xni)

 

EXP% = Expected % = 100 * ( A + B/2 ) / ( A+B+C )

 

If OBS%<EXP% then the local data are more random than the model predicts.

If OBS%>EXP% then the local data are more predictable than the model predicts.