On-Demand Lecture Watching System Using Various Actions of Student Characters to Maintain Concentration

AI summary

Overview

Research area: Human-Computer Interaction, specifically educational technology, virtual agents/characters, and on-demand (recorded) lecture systems.

Technical level: Intermediate. The paper combines a moderately technical system build (Unity, a naked-eye 3D display, motion capture) with accessible educational-psychology concepts, and its results are reported as behavioral and questionnaire outcomes rather than as algorithm detail.

Scope: The paper develops and evaluates an on-demand lecture-watching system that surrounds the viewer with 22 animated student characters in a virtual classroom shown on a naked-eye 3D display, and tests whether two different schedules of character actions affect learners' concentration and notetaking.

What This Paper Is About

Online and on-demand lectures are convenient, but they suffer from a loss of concentration because students do not feel the presence of classmates. The authors' goal is to restore that sense of copresence by placing a group of 3D student characters in a virtual classroom who behave like real classmates, and to find out which pattern of character actions best helps learners stay focused during a long recorded lecture.

Key Contributions

1. A complete on-demand lecture-watching system built in Unity 2021.3.24f1 and displayed on a Sony ELF-SR1 naked-eye 3D display, containing a lecture screen, an instructor character, and 22 student characters capable of positive, negative, and neutral actions.
2. Two designed action modes for long lectures: Stable Mode (characters stay mostly positive) and Dynamic Mode (characters stay mostly negative and switch to positive only around key lecture segments), each defined with specific timings, transition durations, and an explicit positivity measure ranging from +22 to -22 across the 22 characters.
3. A classroom evaluation with 50 undergraduate participants that measured questionnaire impressions, a 19-question mini-test, the number of notes written (split into key-segment and non-key-segment notes), and torso posture recorded and analyzed with OpenPose v1.7.0.
4. Evidence that the two modes trade off different benefits: Stable Mode promotes notetaking and helps on difficult questions, while Dynamic Mode reduces backward leaning over time, suggesting that including negative actions can support physical engagement even though it does not increase notetaking.

Main Findings

• Both modes felt natural and copresent: Participants rated many questionnaire items favorably, including "I felt like I was in class with other students," and scored high on item 3 ("The characters' actions seemed natural") in both modes, indicating realism was preserved across the 33-minute lecture.
• Dynamic Mode felt less positive: A Wilcoxon rank sum test showed a significant difference between modes on item 4 ("I felt the atmosphere was positive"), with the greater amount of negative action in Dynamic Mode producing a less positive impression.
• No difference in test scores: Mini-test scores showed no statistically significant difference between Stable and Dynamic Mode (Wilcoxon rank-sum test). The overall average was 15.7 out of 19 (82.4%), which the authors interpret as a suitable difficulty for a single class session.
• Stable Mode increased notetaking once non-notetakers were excluded: Total notes showed no significant difference initially, but 16 participants (8 in each mode) wrote no notes at all. After excluding them, the Stable Mode group took significantly more notes, with the gap especially evident for key-segment notes; non-key-segment notes showed no difference.
• Notetakers in Stable Mode did better on hard questions: A mode-by-notetaking split (Stable/notes, Stable/no notes, Dynamic/notes, Dynamic/no notes) analyzed with a Kruskal-Wallis test with Bonferroni corrections found no overall significant differences, but a question-by-question analysis showed that on more difficult items (lower average accuracy), notetakers in Stable Mode achieved notably higher scores.
• Dynamic Mode reduced backward leaning: In both modes, participants who took notes kept a more upright posture while those who did not leaned back more. Between about 12 and 20 minutes, participants in Stable Mode who did not take notes showed marked backward leaning, suggesting decreased engagement; in Dynamic Mode, participants leaned less regardless of whether they took notes.
• Opposite outcome from the authors' assumption: The authors expected Dynamic Mode's well-timed positive actions to prompt notetaking more effectively, but Stable Mode instead matched or exceeded it on notetaking, while Dynamic Mode's advantage appeared in posture instead.

Methodology in Plain English

The researchers first ran a preliminary experiment to learn how to make a group of student characters look natural. Participants judged the timing of action changes, the number of distinct action types, and where negative-action characters (such as sleeping) were placed. Three insights emerged and were built into the final system: more varied actions looked more natural, putting negative-action characters at the back of the classroom looked more realistic while keeping a generally positive impression, and slightly mistimed actions still felt natural.

The main system places 22 student characters in a virtual classroom rendered on a naked-eye 3D display, so viewers see stereo depth without wearing a headset. Characters perform positive actions (notetaking, nodding), negative actions (sleeping, looking away, leaning on an elbow), and neutral actions (leaning forward, sitting upright). Each action has between one and three variations. For notetaking, a character looks at the on-screen slides for 0.5 to 1.0 seconds and then at its notebook for 3.0 to 4.0 seconds. Actions were recorded with an Azure Kinect DK and synchronized to the lecture timeline, and each character's action sequence (type and timing) was scheduled in a CSV file so longer videos could be managed.

In Stable Mode, nearly all characters stay positive for most of the lecture, with half the front row switching between notetaking and nodding every 18 to 30 seconds, followed by a similar switch in the back row; during an embedded video segment the characters first switch to leaning forward or sitting upright and then change actions randomly at five equally spaced intervals. In Dynamic Mode, most characters stay negative, begin transitioning to positive actions two seconds before each key segment, hold them until the segment ends, and then gradually revert; all characters lean forward for 4 seconds and move from the far edges of the back row to the center of the front row over 60 seconds before returning to negative actions. Across the lecture, Dynamic Mode characters spent 24.8% of the time in positive actions, 42.1% in transition, and 33.1% in negative actions.

To gauge the overall tone of each mode, the authors assign each character +1 for a positive action, -1 for a negative action, and 0 otherwise, then sum across the 22 characters, giving a possible range of +22 to -22. In both modes, right after the lecture starts, front-row characters lean forward and sit up straight while back-row characters lie down, look away, or put elbows on their desks, performed at random per row. When many characters transition together, the transition takes 2 to 4 seconds with randomly shifted timing within that window to look natural.

The evaluation ran from December 12 to December 22, 2023, in a university School of Engineering "Computer Graphics" course with 50 undergraduate participants (39 men, 11 women, average age 20.8), randomly assigned 25 to each mode. Participants watched a 33-minute prerecorded lecture titled "Computer Graphics Extra" consisting of 29 slides, with an A4 handout of the slides for notetaking. They then had 3 minutes to review their notes, after which the handouts were collected, answered a 19-question multiple-choice mini-test via Microsoft Forms, and completed a lecture impression questionnaire on the same platform. Participants were filmed from their left side, and they were told that posture would not affect their grades. Four measures were used: a 10-item impression questionnaire on a 7-point Likert scale (1 = strongly disagree to 7 = strongly agree), the 19-question mini-test based on content not printed on the handouts, the number of notes counted per slide and classified as key-segment or non-key-segment based on whether the topic was tested, and torso angle computed from the video with OpenPose v1.7.0, where angles greater than 90 degrees indicated leaning back.

Why This Matters

Impact on research. The paper extends prior work by the same group, which had shown that two nodding student characters overlaid on a lecture video improved mini-test scores and that alternating nodding with negative actions temporarily improved audience retention. It advances that line by showing that on a naked-eye 3D display, the choice between an always-positive group and a mostly-negative group that perks up at key moments produces different benefits rather than a single winner, and by demonstrating that the effect persists across a 33-minute lecture. It also links the design question to the literature on social facilitation, social presence, communication fields, and virtual audiences used in concerts and live music.

Real-world applications:

• On-demand and recorded university lectures where students watch alone at home and need a substitute for the presence of classmates.
• HyFlex and blended courses that combine face-to-face sessions with recorded content, where the recorded portion currently lacks any sense of a shared classroom.
• Corporate training and e-learning platforms that want to sustain attention through long video modules without relying on intrusive pop-ups or quizzes.
• Schools exploring xR, including VR-based lessons such as those at N High School in Japan, where a naked-eye 3D display offers stereoscopic depth without the VR sickness, constriction, or bulk of a headset over long viewing periods.

Industry relevance. The approach uses off-the-shelf components (Unity, a commercially available autostereoscopic display, and a Kinect sensor for action capture) and a simple CSV-based action scheduler, which makes it plausible to integrate into existing lecture-recording and learning-management pipelines. Naked-eye 3D displays are already used in digital signage, game display, education, CAD, and product design, and the paper argues that this same capability lets students keep taking notes or consulting other materials while watching in stereo.

Future Directions

• Applying the approach to synchronous online lectures, for example by having real students appear as avatars alongside the CG classmates to reinforce copresence during live sessions.
• Building a VR-based version to deepen immersion, while studying whether character animations remain convincing at life-size scale, since the authors note that unrealistic motion at full scale would be a problem.
• Testing with harder lecture content, because the advantage of Stable Mode notetakers appeared only on the more difficult mini-test items, and the authors suggest the cues from frequent positive actions might help more if the material were more difficult overall.
• Resolving the trade-off between the two modes, since Stable Mode encouraged notetaking but some non-notetakers leaned back between about 12 and 20 minutes, while Dynamic Mode kept people from leaning back but did not increase notetaking as effectively, leaving open how to combine or adapt modes within a single lecture.

Target Audience

Researchers and graduate students in human-computer interaction, educational technology, and virtual agent design; instructors and instructional designers responsible for recorded or HyFlex courses; developers of e-learning and corporate training platforms looking for non-intrusive ways to sustain attention; and anyone working with naked-eye 3D or xR displays in educational settings. Readers primarily interested in cognitive science theory will find the behavioral measures useful, while readers looking for detailed statistical reporting should note that effect sizes and p-values are not reported in the available content.

Authors’ abstract

Since the COVID-19 pandemic, online lectures have spread rapidly and many students are satisfied with them. However, one challenge remains the loss of concentration due to the lack of students' copresence. Our previous work suggests that presenting 3D characters with appropriate actions has the potential to improve concentration in online lectures. Nevertheless, an effective combination of actions has not yet been identified. In this study, we developed a lecture watching system that presents a 3D virtual classroom using a naked-eye 3D display. The system includes student characters that show copresence with various actions such as nodding, notetaking, and sleeping. An evaluation experiment was conducted with two conditions; (1) student characters perform only positive actions and (2) both positive and negative actions. The results, analyzed using posture and notetaking behavior as key indicators, suggest that the system can help to maintain concentration when the student characters perform both positive and negative actions, rather than only positive ones. These findings provide promising strategies for maintaining student focus in on-demand lectures and contribute to the development of more effective online education systems.