Models and Programs for Coefficients of Reliability of
a Test of Ordinal Categorical Items
Yasuharu Okamoto
These programs calculate the coefficient of reliability for ordinal categorical (polytomous) items, which indicates strength of the relationship of observed scores and true values (Okamoto, 2016). There are two ways of thinking about true values. In this website, true values are with respect to the concept that is measured. The other way is to think true values as averages of observed scores, i.e. averages of categorical values. However, psychological scales are used to measure concepts, that is, the purpose of measurement is not to estimate expectations of categorical scores, but to infer true values with respect to the concept.
Okamoto (2016) showed that other current coefficients, e.g. alpha coefficient, tend to overestimate relationship of observed scores and true values.
Programs
This document (Adobe PDF) explains how to use the program.
The files of the programs are archived in this file (*.ZIP).
Models
Let 
be an
unobserved continuous value on item 
of a
person. A single factor model for is set
as follows: (Equation numbers follow those in Okamoto (2016))
where 
is a
common factor with the standard normal distribution of a person, 
is a
factor loading, 
is a
position parameter, and 
is an
error that has normal distribution with mean 0 and variance 
.
The observed response 
on
item , based on , is made according to the rule
where 
s are category boundaries for response , and
An observed score 
can be
obtained as the sum of s, that is,
Reliability coefficient 
, which denotes a test’s precision as coefficient 
for
underlying latent variables s, is proposed by Okamoto (2013). That is, is
given by
Coefficient alpha 
is
also calculated for ordinal categorical items as follows:
A parallel
test 
of is
represented as follows:
In this study, the reliability coefficient defined by the correlation
coefficient of parallel tests and 
is
denoted by 
:
Set the following regression equation
The reliability coefficient as predictive power, denoted by 
in
this study, is given by
gives
the coefficient of reliability, that indicates strength of the relationship of
observed score and
true value .
The difference 
indicates the loss of information by
categorization (discretization).
Constraints to identify parameter values
Items with More than Two
Categories
Set
Then, we have
Equation 12 implies that an origin and a unit of the scale are arbitrary.
For the origin and the unit, Okamoto (2013)
sets
Binary Items
For binary items, Constraint 14 cannot be employed, because a single
category boundary 
bisects the continuum of the latent
variable . In the case of binary items, Equation 12 becomes
Because of arbitrariness of origin, set
then we have
Put
and then probabilities for binary items can be given by parameters 
s and 
s, instead of s, s, and 
s. That is, for binary items, the unit for each
item is set
so that 
, and the origin is set so that 
. Hence, we can estimate reliability coefficients , , or for
binary items. But, reliability coefficient cannot
be calculated from these parameters because item s do not have the same common unit.
Note that
and
where
The formulae used to
calculate 
, 
, and 
First, we have
because, by assumption,
We have
By definition of expectation,
We have
where 
is the
standard normal distribution function and 
is the
cumulative distribution function of .
It is assumed that
We have
For the terms in Equation A.5, we have
and
We have
Note that Equation A.8 assumes 
.
Since
,
The numerator of Equation A.1 can be given as
Using Equations A.1 to A.9,
can be
calculated with parameter values s, s, s and s.
By definition, is
given by
where is a
parallel test of .
We have
and
Probability 
is
given by
Note that since is a
parallel test of , 
might
be equal to 
, but in Equation A.8, we assume that .
can be
calculated by Equations A.2, A.3, A.10, A.11, A.12, and A.13.
Coefficient is
given by
We have
The two terms of Equation A.15 can be calculated as follows:
and
Hence, coefficient can be
calculated by Equations A.2, A.14, A.15, A.16, and A.17.