Extended Rater Representations (Elliott & Buttery, 2022)

Elliott & Buttery (2022) propose Extended rater representations. These are implemented in Elliott's Python-based RaschPy software.

 

According to Section 6.2.1 of the RaschPy manual, there are 4 extended rater representations: Global, Item, Thresholds and Matrix. These can be implemented in several ways using Facets. Here are suggestions:

 

---

 

1.Global: raters, items, and ratees have constant measures. All share the same Andrich rating scale.

 

The MFRM measurement model:

ratee ability - rater severity - item difficulty - Andrich rating scale -> observed rating

 

Facets specification:

Model = ?, ?, ?, R

where facet 1 = ratees, facet 2 = raters, facet 3 = items

 

Example: Creativity - Guilford.txt

 

---

 

2.Item: each rater has a different severity for each item.  Items and ratees have constant measures. All share the same Andrich rating scale.

 

2a. Reporting rater-item interactions without altering the "1. Global" measures:

 

The MFRM measurement model:

ratee ability - overall item difficulty - overall rater severity - Andrich rating scale -> observed rating

Bias-interactions: (rater severity for the item)

 

Global model reporting bias-interactions:

Facets specification:

Model = ?, ?B, ?B, R

where facet 1 = ratees, facet 2 = raters, facet 3 = items

 

Example: Creativity - Guilford.txt with Model = ?, ?B, ?B, R

 

2b. Rater-item interactions alter measures:

 

The MFRM measurement model:

ratee ability  - overall rater severity - overall item difficulty- (rater severity for the item) - Andrich rating scale -> observed rating

 

Facets specification:

Model = ?, ?, ?, ?, R

where facet 1 = ratees, facet 2 = raters (anchored at 0), facet 3 = items, facet 4 = rater+item (group-anchoring items at 0)

 

Example: Guilford-extended-rater.txt

 

---

 

3.Thresholds: each rater has a different set of Andrich rating scale thresholds (= Partial Credit Model for raters).  Items and ratees have constant measures.

 

The MFRM measurement model:

ratee ability - overall rater severity - overall item difficulty - (PCM thresholds by raters)  -> observed rating

 

Facets specification:

Model = ?, #, ?, R

where facet 1 = ratees, facet 2 = raters, facet 3 = items

 

Example: Creativity - Guilford.txt with Models= #,?,?,R9 ; judges, examinees, items

 

---

 

4.Matrix (Marginal=True): each rater+item has a different severity for each item and each rater has a set of Andrich rating scale thresholds (= Partial Credit Model for raters).  Items and ratees have constant measures

 

4a. Reporting rater-item interactions without altering the "3. Threshold" measures:

 

The MFRM measurement model:

ratee ability - overall rater severity - overall item difficulty -  (PCM thresholds for each rater)  -> observed rating

Bias-interactions: (rater severity for the item)

 

Facets specification:

Model = ?, ?#B, ?B, R

where facet 1 = ratees, facet 2 = raters, facet 3 = items

 

Example: Creativity - Guilford.txt with Model = ?, #B, ?B, R

 

4b. Rater-item interactions alter measures:

 

The MFRM measurement model:

ratee ability  - overall rater severity - overall item difficulty - (rater severity for the item) - (PCM thresholds for each rater)  -> observed rating

 

Facets specification:

Model = ?,#, ?,?, R

where facet 1 = ratees, facet 2 = raters (anchored at 0, PCM), facet 3 = items, facet 4 = rater+item (group anchoring items at 0)

 

Example: Guilford-extended-rater.txt with Model = ?, #, ?,?, R

 

---

 

5.Matrix (Marginal=False): each rater+item has a different severity for each item and each rater+item combination has a set of Andrich rating scale thresholds (= Partial Credit Model for raters+items).  Items and ratees have constant measures

 

4a. Reporting rater-item interactions without altering the "3. Threshold" measures:

 

The MFRM measurement model:

ratee ability - overall rater severity - overall item difficulty -  (PCM thresholds for each rater+item)  -> observed rating

Bias-interactions: (rater severity for the item)

 

Facets specification:

Model = ?,#B, #B, R

where facet 1 = ratees, facet 2 = raters, facet 3 = items

 

Example: Creativity - Guilford.txt with Model = ?, #B, ?B, R

 

4b. Rater-item interactions alter measures:

 

The MFRM measurement model:

ratee ability  - overall rater severity - overall item difficulty - (rater severity for the item) - (PCM thresholds for each rater+item)  -> observed rating

 

Facets specification:

Model = ?,?, ?,#, R

where facet 1 = ratees, facet 2 = raters (anchored at 0), facet 3 = items, facet 4 = rater+item (PCM, group anchoring items at 0)

 

Example: Guilford-extended-rater.txt with Model = ?,?, ?,#, R

 

 

---

 

Elliott, M. (2023). RaschPy.

Elliott, M., & Buttery, P. J. (2022a). Extended rater representations in the many-facet Rasch model. Journal of Applied Measurement, 22 (1), 133–160.

 

---