Generalizing Classification of Pilot Workload: Transfer Learning versus a JEPA-Inspired Transformer Architecture

Abstract

Within the context of learning, there poses difficulty when objectively measuring human performance. In this work, we investigate the evaluation of human performance via its relation to the individual's mental capacity by classification of cognitive load within the domain of aviation. By utilizing a mixed virtual and physical flight simulation environment in conjunction with biometric sensing, we create and evaluate the predictive capabilities of a Joint-Embedding Predictive Architecture (JEPA) and compare the architecture and results to traditional methods for transfer learning and domain adaptation. We find that our JEPA inspired architecture can achieve more than 70% accuracy of cognitive workload, compared to the 63% and 56% accuracies of traditional transfer learning methods. Through this foundation, we have made advancements in multi-modal and multi-task learning to classify various features across numerous pilots, operators, and novices within aviation. Our predictive model can automate the evaluation of cognitive load, enabling creation of generalizing features even when labeled examples are scarce.

Authors

• Barnett, Naim Embry-Riddle Aeronautical University
• Nagrecha, Shivani Embry-Riddle Aeronautical University
• Glover, Morgan Embry-Riddle Aeronautical University
• Harper, Clayton Embry-Riddle Aeronautical University
• Wilson, Justin Embry-Riddle Aeronautical University
• Maher, James Embry-Riddle Aeronautical University
• Larson, Eric C Embry-Riddle Aeronautical University

Keywords

• Cognitive Load
• Machine Learning
• Biometrics
• Artificial Intelligence and Robotics
• Aviation and Space Education
• Graphics and Human Computer Interfaces