Effect size

There are several definitions of "effect size" - https://en.wikipedia.org/wiki/Effect_size

 

A common one is:

Effect size = (treatment B mean effect - treatment A mean effect) / (treatment A sample S.D.).

 

In Facets this would be:

Effect size = (Task B measure - Task A measure) / (S.D. of person measures)

 

Also:

 

Cramer's V converts a chi-squared into on effect size.

 

For an individual rater:

Here is one similar to Cohen's D:

Effect size = (rater leniency - mean rater leniency)/rater leniency S.D. of sample

 

One similar to Glass Delta Δ is

Effect size = (rater leniency - mean rater leniency)/rater latency S.E. of this rater

 

 

Derivation of Cohen's d with joint (pooled) S.E.s

 

Cohen's d: quoting Wikipedia: "difference between two means divided by a pooled standard deviation for the data"

 

So, in a simple case, let's assume the two sample sizes are the same, n, so that the combined sample is 2*n = N

 

The sample S.D. is used in Cohen's d, e.g., https://scientificallysound.org/2017/06/29/cohens-d-and-effect-size/

 

Sample S.D. for one sample = √(sample variance / (n-1)).

S.E. of the sample mean = √(sample variance / (n*(n-1))) = S.D. / √(n)

Rearranging, SD = √ (n) * SE

 

Let's add up the variances for two samples: pooled variance  = (n-1)*SD1² + (n-1)*SD2²

 

pooled S.D. = √ ( (n-1)*SD1² + (n-1)*SD2²) / (n+n-2))

= √ (( SD1² + SD2²) / 2 )

= √ ((n*SE1² + n*SE2²) / 2)

pooled S.D.= √ (SE1² + SE2²) * √(n/2)

 

but n = N/2  where N is the combined sample

= √ (SE1² + SE2²) * √(N/4)

 

Facets: t = (M1-M2) / √ (SE1² + SE2²)

Cohen's d = (M1-M2) / pooled S.D.

Cohen's d = Facets t * √ (SE1²+ SE2²)  / pooled S.D.

= t * √ (SE1²+ SE2²)  / ( √ (SE1² + SE2²) * √(N / 4)

= t * 2 / √(N)

 

So, for Facets, when there is a joint S.E., its t and its d.f., the approximation becomes:

Cohen's d = 2 * t /√(d.f.)