Reliability of a Test of Categorical (Polytomous) Items

Japan Women’s University Journal: Faculty of Integrated Arts and Social Sciences, 2016, 27, 113-122.

Reliability Coefficients for Ordinal Categorical Items:

Actual Relation of Observed and True Scores

Yasuharu Okamoto

Abstract

The coefficient for ordinal categorical items that represents the strength of the relation between observed scores and

true values is proposed and compared with reliability coefficients by simulation.

For continuous items, the relational strength of observed scores and true values can be represented by reliability coefficients.

However, for ordinal categorical items, an observed categorical response is modeled according to Thurstone:

This type of model requires a coefficient for ordinal categorical items to be provided differently

from that for continuous items. The proposed coefficient for ordinal categorical items is

a coefficient of determination for a regression model, which represents the relation between observed scores and true values.

Results of simulations of coefficients for ordinal categorical items indicate that

reliability coefficients, the correlation coefficient of parallel tests, and the coefficient alpha overestimate

actual relationships of observed scores and true values, which the proposed coefficient adequately represents.

Keywords

reliability coefficient, ordinal categorical items, coefficient of determination, regression model