Subsets: Nested facet designs: remedying ambiguities

This is for 32-bit Facets 3.87. Here is Help for 64-bit Facets 4

Here is a typical nested design. There are 7 prompts, but candidates only respond to one. From the measurement perspective, we cannot tell whether Mary scored higher than George was because Mary is the more able or because she answered the easier prompt.

 

There are several reasonable measurements solutions to this problem.

 

(A) Random Equivalence of Nested Subsets.

 

If the candidates are assigned to each prompt randomly, then the candidate-prompt subsets can be thought of as randomly equivalent. This implies that we can assert that the subsets have the same mean. This is implemented with group-anchoring.

 


 

A little word-processing of Table 7 can turn the "disjoint subset number" into the "group number":

 

Table 7.3.1  rater Measurement Report  (arranged by MN).

 

--------------------------------------------------------------------------------

| Obsvd   Obsvd  Obsvd  Fair-M|        Model | Infit      Outfit   |           |

| Score   Count Average Avrage|Measure  S.E. |MnSq ZStd  MnSq ZStd | Num rater |

--------------------------------------------------------------------------------

|    124     18     6.9   6.64|  -2.49   .38 |  .4  -1     .4  -1  |  74 Mary  | in subset: 12

|    123     18     6.8   6.58|  -2.35   .31 |  .4  -1     .4  -1  |  83 Fred  | in subset: 14

|    124     18     6.9   6.40|  -2.05   .39 |  .4  -1     .4  -1  | 430 Jose  | in subset: 29

|    121     18     6.7   6.31|  -1.93   .27 |  .4  -2     .4  -2  | 180 Chin  | in subset: 27

 

becomes

 

Labels=

.....

3,Raters,G

 74, Mary, 0, 12

 83, Fred, 0, 14

430, Jose, 0, 29

180, Chin, 0, 27

 

This can be done with the Output files menu, Subset file.

 


Example of a specification file for group-anchoring:

 

Facets=3  ; Prompt, Candidate, Rater

Positive=2  ; Candidates are oriented: larger score=>greater message

Noncenter=1  ; Prompts float

Labels=

1,Prompts

1=Prompt 1

2=Prompt 2

...

7=Prompt 7

*

2,Candidates,G  ; group-anchoring is requested: label groups by their prompt number.

5433=George,0,3 ; George, candidate 5433, is anchored at 0, as part of group 3

7629=Mary,0,2  ; Mary, candidate 7629, is anchored at 0, as part of group 2

3245=Anne,0,4  ; Anne responded to prompt 4

6751=Jose,0,2  ; Jose responded to prompt 2

.....

4765=Zoe,0,5  ; Zoe responded to prompt 7

*

3,Rater   ; The raters rated all prompts. More than one rater per candidate response.

1,Dr. Jones

2,Ms. Smith

3,Sra. Lopez

....

*

Data=

2,3,2,4   Prompt 2, Mary: Ms. Smith gives rating of 4.

......

 

Facet 2, Candidates, is specified with ",G". This implements group-anchoring. The members of each group are allowed to float relative to each other, but their mean measure is constrained to be the mean of their anchor values. In this case, the group means of candidates are set to zero. The differences between the groups are forced into the prompt difficulties. High scoring subsets answered easy prompts. Low scoring groups answered more challenging prompts.

 

(B) Equally difficult or known measures for the prompts.

group-anchoring is not used, but the prompts are anchored.

 

Facets=3  ; Prompt, Candidate, Rater

Positive=2  ; Candidates are oriented: larger score=>greater message

Noncenter=2  ; Candidates float

Labels=

1,Prompts,A

1=Prompt 1,0  ; Prompts all anchored to be the same difficulty

2=Prompt 2,0

...

7=Prompt 7,0

*

2,Candidates

5433=George

7629=Mary

3245=Anne

6751=Jose

.....

4765=Zoe

*

3,Rater   ; The raters rated all prompts. More than one rater per candidate response.

1,Dr. Jones

2,Ms. Smith

3,Sra. Lopez

....

*

Data=

2,3,2,4   Prompt 2, Mary: Ms. Smith gives rating of 4.

......

 

The prompts will all have the same measure of zero. This is equivalent to ignoring the prompt facet for measurement, but each prompt is reported with fit statistics commenting on the consistency with which it is rated.

 

(C) Candidates choose prompt.

In this situation, less able candidates may choose one prompt, "My day at the zoo", but more able ones may choose another, "Einstein's theory of relativity." This requires further information beyond the data.

 

"Virtual equating" is a useful post-hoc technique if the candidate essays and content experts are available. Each candidate-prompt subset is analyzed separately. For each prompt, a set of consistently-rated pieces of work is chosen, located about half-logit apart along the variable. These form substantive "prompt rulers". The experts then slide these rulers relative to each other to align pieces of work representing the same latent ability.

This provides the anchor values for the prompts.

 

(D) Equivalent elements.

If two elements are functionally the same, then the principle of "Virtual equating" applies. A convenient way of assigning two different elements the same element number is the "Target element number" in Labels=.

 

(D) Equal measures for equal candidate scores.

A specific candidate raw score is chosen as the banchmark value. All candidates with this score are anchored at the same measure, All other measures (raters, items, other candidates, etc.) align with this measure.


Help for Facets Rasch Measurement and Rasch Analysis Software: www.winsteps.com Author: John Michael Linacre.
 

Facets Rasch measurement software. Buy for $149. & site licenses. Freeware student/evaluation Minifac download
Winsteps Rasch measurement software. Buy for $149. & site licenses. Freeware student/evaluation Ministep download

Rasch Books and Publications
Invariant Measurement: Using Rasch Models in the Social, Behavioral, and Health Sciences, 2nd Edn, 2024 George Engelhard, Jr. & Jue Wang Applying the Rasch Model (Winsteps, Facets) 4th Ed., Bond, Yan, Heene Advances in Rasch Analyses in the Human Sciences (Winsteps, Facets) 1st Ed., Boone, Staver Advances in Applications of Rasch Measurement in Science Education, X. Liu & W. J. Boone Rasch Analysis in the Human Sciences (Winsteps) Boone, Staver, Yale
Introduction to Many-Facet Rasch Measurement (Facets), Thomas Eckes Statistical Analyses for Language Testers (Facets), Rita Green Invariant Measurement with Raters and Rating Scales: Rasch Models for Rater-Mediated Assessments (Facets), George Engelhard, Jr. & Stefanie Wind Aplicação do Modelo de Rasch (Português), de Bond, Trevor G., Fox, Christine M Appliquer le modèle de Rasch: Défis et pistes de solution (Winsteps) E. Dionne, S. Béland
Exploring Rating Scale Functioning for Survey Research (R, Facets), Stefanie Wind Rasch Measurement: Applications, Khine Winsteps Tutorials - free
Facets Tutorials - free
Many-Facet Rasch Measurement (Facets) - free, J.M. Linacre Fairness, Justice and Language Assessment (Winsteps, Facets), McNamara, Knoch, Fan
Other Rasch-Related Resources: Rasch Measurement YouTube Channel
Rasch Measurement Transactions & Rasch Measurement research papers - free An Introduction to the Rasch Model with Examples in R (eRm, etc.), Debelak, Strobl, Zeigenfuse Rasch Measurement Theory Analysis in R, Wind, Hua Applying the Rasch Model in Social Sciences Using R, Lamprianou El modelo métrico de Rasch: Fundamentación, implementación e interpretación de la medida en ciencias sociales (Spanish Edition), Manuel González-Montesinos M.
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch Rasch Models for Measurement, David Andrich Constructing Measures, Mark Wilson Best Test Design - free, Wright & Stone
Rating Scale Analysis - free, Wright & Masters
Virtual Standard Setting: Setting Cut Scores, Charalambos Kollias Diseño de Mejores Pruebas - free, Spanish Best Test Design A Course in Rasch Measurement Theory, Andrich, Marais Rasch Models in Health, Christensen, Kreiner, Mesba Multivariate and Mixture Distribution Rasch Models, von Davier, Carstensen
As an Amazon Associate I earn from qualifying purchases. This does not change what you pay.

facebook Forum: Rasch Measurement Forum to discuss any Rasch-related topic

To receive News Emails about Winsteps and Facets by subscribing to the Winsteps.com email list,
enter your email address here:

I want to Subscribe: & click below
I want to Unsubscribe: & click below

Please set your SPAM filter to accept emails from Winsteps.com
The Winsteps.com email list is only used to email information about Winsteps, Facets and associated Rasch Measurement activities. Your email address is not shared with third-parties. Every email sent from the list includes the option to unsubscribe.

Questions, Suggestions? Want to update Winsteps or Facets? Please email Mike Linacre, author of Winsteps mike@winsteps.com


State-of-the-art : single-user and site licenses : free student/evaluation versions : download immediately : instructional PDFs : user forum : assistance by email : bugs fixed fast : free update eligibility : backwards compatible : money back if not satisfied
 
Rasch, Winsteps, Facets online Tutorials

Coming Rasch-related Events: Winsteps and Facets
Oct 21 - 22 2024, Mon.-Tues. In person workshop: Facets and Winsteps in expert judgement test validity - UNAM (México) y Universidad Católica de Colombia. capardo@ucatolica.edu.co, benildegar@gmail.com
Oct. 4 - Nov. 8, 2024, Fri.-Fri. On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
Jan. 17 - Feb. 21, 2025, Fri.-Fri. On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
May 16 - June 20, 2025, Fri.-Fri. On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
June 20 - July 18, 2025, Fri.-Fri. On-line workshop: Rasch Measurement - Further Topics (E. Smith, Facets), www.statistics.com
Oct. 3 - Nov. 7, 2025, Fri.-Fri. On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com

 

Our current URL is www.winsteps.com

Winsteps® is a registered trademark