Go to main content Go to main menu
Left Menu

Providing Accurate Diagnostic Feedback using Extant Test Data

Seminars
Date March 20, 2018
Time 13:30 - 14:30
Speaker
Mr Yan Sun
Venue
Room 665 (Meeting Room), Meng Wah Complex, HKU

EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT COLLOQUIUM SERIES

Providing Accurate Diagnostic Feedback using Extant Test Data

Yan Sun
The State University of New Jersey

March 20, 2018 (Tuesday)
13:30 – 14:30
Room 665 (Meeting Room), Meng Wah Complex, HKU


Abstract
Cognitive diagnosis models can provide information about test takers’ mastery and nonmastery of finer-grained skills or attributes. The mapping of test items by skills given in the Q-matrix partly determines the quality of subsequent actions. In this study, the possibility of using extant large-scale math test to provide formative diagnostic information is explored. Four test forms each with 60 items are considered. The attributes and Q-matrices are defined and developed by content experts, and are empirically validated under the generalized deterministic input, noisy and gate model framework. Due to the large number of attributes involved, an alternative estimation procedure, (i.e., the accordion approach) is adopted to make parameter estimation feasible. In the current study, three domains are identified, and three corresponding Q-matrices are built for each test form. Results show that the Q-matrices suggested by the validation process and those built by content experts have high agreements. However, the resulting posterior distribution shows uneven classification accuracies among examinees. One possible solution offered for examinees who lack classification precision is to administer additional items adaptively. A simulation study is designed to examine what types of and how many additional items are needed to be administered in a cognitive diagnosis computerized adaptive testing context for all students to reach a prescribed level of classification accuracy. This work represents a first step in tapping the potential of extant large-scale tests for diagnostic purposes.

About the Speaker
Yan Sun is a Ph.D. candidate at the Graduate School of Education at Rutgers, The State University of New Jersey, USA. He is currently an honorary research associate in the Faculty of Education at The University of Hong Kong. His research interests are mainly on advanced psychometric models that can be used to inform teaching and learning.

Everyone is welcome to attend!

If interested, please confirm your attendance by sending an email to
kpsantos@hku.hk.

TOP
HKU HomeHKU PortalFISFaculty PortalContact UsPrivacy Policy