Extended Abstract—Immersive Education with Historical Characters: Conversational MetaHuman Based on Large Language Model, Speech Recognition and Generation

Abstract

With the advancement of technologies in computer graphics, large language models, and speech processing, the development of interactive characters for historical education has become increasingly feasible. This work-in-progress paper presents a functional prototype that integrates frameworks such as Unreal Engine 5, MetaHuman and Convai to develop an interactive system for engaging with historical figures. This system provides an accessible solution for educational institutions, particularly those with limited resources, by minimizing development efforts while maintaining a high level of interactivity, immersion, and customizability. Users can engage in conversational interactions with historically accurate figures, who respond in real-time with facial animations, body language, and natural speech. These characters are designed to generate meaningful dialogue based on databases of well-documented historical knowledge to enhance educational experiences and effectiveness. This approach offers a novel method for cultural and historical education, providing an immersive interface that allows users to engage with history in a more personal and immediate manner. The paper also includes a discussion of potential future enhancements to improve user engagement and learning outcomes. This work demonstrates the potential of integrating contemporary technologies to develop dynamic, accessible, and impactful tools for historical education, paving the way for more engaging and effective methods of experiencing history.