Up

Proceedings of the 15th Annual Conference of the Japan Association for Research on Testing, 2017,152|155.

Three Aspects of Reliability in Item Response Theory

Visual representations of reliability

›Yasuharu Okamoto

Faculty of Integrated Arts and Social Sciences, Japan Womenfs University

 

1. Introduction

    Reliability has mainly been discussed in classical test theory (CTT), and coefficients have been defined according to three aspects, precision, stability, and predictive power. Precision comes from the ratio of the variance of true scores to that of observed scores and is considered the mathematical definition of reliability. Stability is denoted by correlation coefficient between two parallel tests. Predictive power is represented by strength of relation between observed values and true values and is obtained by the regression modelfs coefficient of determination for observed values and true values as dependent and predictor variables, respectively. These three coefficients of reliability coincide in the case of continuous items. However, in the case of categorical items, they differ. Results of simulation (Okamoto, 2016) showed that alpha coefficient tends to be larger than strength of relation, i.e., coefficient of determination, between true values and observed values in the case of categorical items. These results indicate the necessity of considering reliability in item response theory (IRT) from the three points of view.

FullText (manuscript)

 

Up