The comparative usability of instructor feedback given in three modes: Screencast, audio, and written
Date
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
This study explores the comparative usability of multimodal feedback, specifically exploring the relative efficiency, effectiveness, and satisfaction usability advantages and disadvantages of audio, screencast, and written feedback or instructor response for students, and it explores the level of satisfaction experienced by instructors when giving feedback in each mode. To conduct this investigation, this study tested four hypotheses based on preceding research. (H1) SUS (System Usability Scale) and product reaction survey results will show that students prefer screencast feedback above audio feedback and audio feedback above written feedback; (H2) screencast feedback will show the highest levels of effectiveness for students and both screencast and audio feedback will show higher levels of effectiveness than written feedback; (H3) screencast feedback will show the higher levels of efficiency for students than audio feedback but written feedback will show the highest levels of efficiency; and (H4) instructors will express higher satisfaction in giving written feedback than they do for audio and screencast feedback.
To test these hypotheses, 20 students were recruited to use one of the three modes of feedback and be observed, surveyed, and retrospectively interviewed about their experiences. In addition, four instructors were also observed, surveyed, and retrospectively interviewed about their experiences giving feedback in one of the three modes. Test results supported H1a, which says that students show a preference for screencast feedback above audio feedback, but H1b was not supported, which says that students did express higher satisfaction for written feedback, at least according to SUS results. H2 showed inconclusive results regarding whether screencast and audio feedback offers users a more effective experience than written feedback, though there were indicators of unique advantages offered by each mode. H3a was supported, indicating efficiency advantages in connection with student use of screencast feedback over audio feedback, but it remains inconclusive whether screencast offers definitive efficiency advantages over written feedback. H4 was well-supported, showing that instructors, despite experiencing some satisfaction and other usability advantages when using screencast or audio feedback, still prefer written feedback.
This study also discovered that both students and instructors initiate or appear to cause user errors, often as a result of being unwilling to or unaware of how to apply feedback received, and sometimes this results from lack of detail or context provided by the instructor. It also asserts that audio feedback does not appear to offer any distinct advantages or affordances over screencast feedback, though a combination of screencast and written feedback is likely to offer students a more ideal usability experience in light of the advantages of both modes. In addition, this study found that cross-purposes between instructors and students are at play in the user scenario, specifically instructor resistance to adopting multimodal feedback methods even though students would prefer that instructors do so.
The primary implications for this study are as follows: (1) Given the apparent usability and learning advantages of using screencast or other multimodal forms of feedback for students and instructors, more needs to be done to investigate how to ameliorate instructor resistance to adoption. (2) Instructors need to take steps to both train their own students how to effectively use and apply feedback but also ensure the feedback they give is contextualized and actionable. (3) Instructors need to take advantage of the affordances of screencast feedback and written feedback to ensure their students receive both more abundant and contextualized feedback as well as sentence-level feedback.