Toward more valid and reliable classroom assessment of digital multimodal composition performances: developing a scoring rubric to assess upper elementary students' digital multimodal book reviews
| Author(s) | Park, Sohee | |
| Date Accessioned | 2019-06-05T12:26:27Z | |
| Date Available | 2019-06-05T12:26:27Z | |
| Publication Date | 2018 | |
| SWORD Update | 2019-04-13T04:01:09Z | |
| Abstract | Digital multimodal composition has drawn attention from literacy scholars and educators as a new type of writing activity that leverages digital technologies and their affordances. However, there is limited research on what and how to assess elementary students’ task-specific digital multimodal composition performances. This mixed-methods study aims to explore the following four research questions in four phases: 1) what are the key domains and criteria that represent the construct of digital multimodal composition? 2) how might these domain and criteria be structured in a task- and grade-specific rubric to evaluate upper-elementary-grade students’ digital multimodal book reviews? 3) to what extent does the use of the proposed rubric display evidence of inter-rater reliability, construct validity and usability, as indicated by raters’ scores and interview feedback? and 4) how does the evidence of inter-rater reliability, construct validity, and raters’ feedback inform the process of rubric revision? In the first phase, a systematic literature review was conducted and a total of five domains and 19 distinguishable criteria were identified from 111 criteria in pre-existing 17 studies. In the second phase, a rubric was created in six steps. In the third phase, rubric validation was conducted by training two raters and collecting both quantitative and qualitative evidence on inter-rater reliability, construct validity, and usability of the developed rubric. The analyses revealed that there were five criteria (i.e., C3-conventions of oral language, C4-conventions of written language, C6-relationship between oral language and written language, C10-quality of summary, and C11-quality of opinion) that seemed to be problematic. In the final phase, the rubric was revised by closely examining the problematic criteria and discussing possible solutions for those through a focus group interview. The findings could contribute to our understanding of digital multimodal composition, rubric development procedures, and the role of the rubric for formative assessments in classrooms. ☐ Keywords: digital multimodal composition, digital multimodal book review, scoring rubric, inter-rater reliability, construct validity, formative assessment, upperelementary grades | en_US |
| Advisor | Karchmer-Klein, Rachel | |
| Degree | Ph.D. | |
| Department | University of Delaware, School of Education | |
| DOI | https://doi.org/10.58088/0ck9-ay49 | |
| Unique Identifier | 1103606337 | |
| URL | http://udspace.udel.edu/handle/19716/24229 | |
| Language | en | |
| Publisher | University of Delaware | en_US |
| URI | ||
| Title | Toward more valid and reliable classroom assessment of digital multimodal composition performances: developing a scoring rubric to assess upper elementary students' digital multimodal book reviews | en_US |
| Type | Thesis | en_US |