Exploring the Real-Time Touchless Hand Interaction and Intelligent Agents in Augmented Reality Learning Applications

Abstract

During the last decade, there has been a surge in research studies exploring the adoption of Augmented Reality (AR) in educational settings. Within these multiple research studies, AR’s capability to extend the teaching and learning environment with augmented 3D learning objects with enhanced interactive capabilities have been demonstrated. This new technology has not been widely adopted in the mainstream but with the recent unprecedented circumstances of COVID-19, there has been an increasing societal willingness to adopt these technologies. AR has been a desirable technology due to its inherent touchless nature which facilitates social distancing at this time but AR applications crucially offer so much more. They can provide interactive functionality through augmentation of the teaching and learning environment within an immersive user experience including 3D interactions with learning objects, gestures, hand interaction, tangible and multi-modal interaction. This paper presents the results of a review of touchless interaction studies in educational applications and proposes the implementation of real-time touchless hand interaction within kinesthetic learning and utilization of machine learning agents. The architecture of two AR applications with real-time hand interaction and machine learning agents are demonstrated within this paper enabling engaged kinesthetic learning as an alternative learning interface.