The development and validation of an assessment that measures middle school students' lunar phase understanding

Date

2009-08

Authors

Sherrod, Sonya E.

Journal Title

Journal ISSN

Volume Title

Publisher

Abstract

Adolescents from across the United States experience instruction targeting lunar phase concepts. However, a single, age-appropriate, multiple-choice instrument that measures understanding of these concepts could not be located. Such an instrument must measure knowledge valued by the 50 state education agencies and the knowledge of lunar phase differences due to hemisphere perspective (ushered in by Internet learning forums). The design of this much needed assessment must take into account common lunar phase misconceptions held by adolescents. This research reports the development and validation of such an instrument called the Comprehensive Moon Phases Assessment (CMPA). State standards and previous research of adolescent misconceptions pertaining to lunar phase concepts framed the CMPA design. The method used to analyze the state standards and identify lunar phase domains was the constant comparative method (Glaser, 1965; Strauss & Corbin, 1994). The taxonomy of rules for writing multiplechoice items compiled by Haladyna and Downing (1989) directed the construction of the CMPA items. The research questions that directed the validation of the CMPA were: (1) Does the CMPA measure the lunar phase constructs found in the state standards? (2) Does the CMPA measure the construct of global lunar perspective? Research questions were answered through expert review and exploratory and confirmatory factor analyses.

An exploratory factor analysis showed 19 items loading onto a 7-factor model, with a confirmatory factor analysis suggesting it to be a model of good fit: 2 = (75, N = 865) = 162.179, p < 0.001, the CFI was 0.964, and the RMSEA was 0.037. While no item received strong scores of five (on a 5-point Likert scale) from all eight reviewers, seven reviewers rated 21 of the 40 CMPA items with scores of four or five. These 21 items were evaluated as valid. Two or more members of the expert review board, using a 5-point Likert scale, ranked 10 of the 19 items that loaded onto a 7-factor model with a score of 3 or lower on content validity. Future research needs to consider feedback from the expert reviewers in developing or revising additional items for the seven factors that were confirmed by the analysis. It is suggested that a revised version of the CMPA be administered to a new sample of middle school students and a factor analysis conducted on the new assessment data.

Description

Keywords

Lunar phase, Assessment of learning, Middle school assessment, Middle school science

Citation