Utilising Video Recordings to Assess Student Pilot Performance: An Exploratory Study during a Simulated Training Flight

Abstract

Verbatim from Embry-Riddle Scholarly Commons. Not paraphrased, not summarized.

In ab initio flight training, the quality of the feedback a flight instructor provides their student is essential to their success. While other high-performance and safety-critical industries have incorporated the use of video recordings into training and assessments, the aviation industry generally has not. There are some studies addressing the potential benefits of video recordings in training professional pilots, but there is a lack of research at the ab initio level. In this study, five flight instructors assessed the performance of student pilots conducting a simulated training flight using a 4-point marking scale. The flights instructors were then tasked with grading the performance of the student pilots from a video recording of the simulated training flight. The grades from the two conditions were compared to determine the extent video recordings of simulated training flights capture the performance markers that flight instructors rely on when evaluating student pilot performance during live assessment. Analysis showed that 60% of the grades remained unchanged between the two conditions and 20.4% experienced a minor change. The remaining 19.6% were major changes in grades, of which 4.2% changed to or from the failing mark of “1”. Cohen’s weighted Kappa showed moderate to substantial agreement of the grades. The results indicate that video recordings appear to capture many of the performance markers that are depended on during a live assessment. Grade discrepancies may be the result of missed performance markers or intra-rater variability. This exploratory study serves as a proof-of-concept demonstration of a video-based assessment methodology and provides a template for future research.

Authors

• Moncion, Bradley Embry-Riddle Aeronautical University
• Cao, Shi Embry-Riddle Aeronautical University
• Lynch, Allison Embry-Riddle Aeronautical University
• Kearns, Suzanne K Embry-Riddle Aeronautical University
• Irving, Elizabeth Embry-Riddle Aeronautical University
• Niechwiej-Szwedo, Ewa Embry-Riddle Aeronautical University

Keywords

• ab initio flight training
• pilot performance
• video recordings
• intra-rater reliability
• Aviation and Space Education
• Educational Assessment, Evaluation, and Research
• Educational Psychology
• Ergonomics
• Human Factors Psychology