1. Introduction

Robotics is considered to be one of the most engaging and tangible subjects in Engineering.^1,2 However, certain kinds of robotics, such as small mobile robots, are more often featured in practical engineering coursework due to safety and cost constraints.³ As a result, engineering students may never get direct practice working with certain types of robotic equipment, such as robotic arms, despite that equipment being vital to modern industry.⁴ In these cases, some instructors have turned to simulations and virtual reality (VR) to provide exposure.^5,6 However, simulations rarely emulate a physical robot perfectly, especially its interaction with the environment.⁷ In addition, the experience of being around a moving, potentially dangerous, robotic arm may be difficult to achieve, even with modern virtual and augmented reality headsets.⁸ As a result, using AR/VR robots in curriculum may be didactically useful, but they may pale in comparison to the incorporation of physical robots from a student perspective.

In this paper, we demonstrate how first- and second-year engineering students can use a two-dimensional (2D) desktop interface to learn robot programming, and subsequently reinforce these concepts by observing a life-size robot arm, programmed by an expert, in operation. The apparent advantages of this set-up are that the student can spend as much time as they need to absorb the underlying concepts of robot programming on a low-cost platform (the 2D interface) with minimal instructor intervention before advancing to the more expensive industrial-scale version of the robot. The industrial experience can be broken into two tiers: first, we have the more expensive physical robot arm, which is the gold standard and is what experts use in industry: in an ideal world, every student would have access to one of these. Second, we include a relatively affordable AR version of the robot, with the main cost being the hardware cost for the headset and the time spent developing the software for it. To demonstrate the usefulness of the system as an effective engineering education tool, it is only natural that we measure what impact the tools have on the students’ learning outcomes.

To this end, we formulated two main research questions that are underpinned by student motivation:

1. What aspects of the active learning programming activities (desktop robot) would the students find motivating?

2. What are the differences between the reinforcement tools (AR robot and physical robot) for motivating the students?

2. Literature Review

2.1. Robots in Education

Robots have been used in education to enhance learning experiences and provide students with hands-on opportunities to develop critical skills such as problem-solving, programming, and teamwork.^9–12 Since a multifaceted and synergistic amalgamation of knowledge from different fields is required to construct a robot, robots provide multiple pathways for learning. They can easily be used as an educational tool in a range of STEM fields, such as computer engineering, computer science, control engineering, electrical engineering, electronic engineering, and mechanical engineering.^13–15 In addition, they provide a tangible platform for students to engage with theoretical concepts in a hands-on manner and elicit enthusiasm in classrooms, making them ideal companions in classroom pedagogy. Their inclusion has been shown to increase knowledge assimilation,¹⁶ student satisfaction,¹⁷ and student engagement,¹⁸ among other things. Further, the application-focused nature of robots lends itself to group settings where students are assigned their own tasks and are given the freedom to work on their subtasks and contribute to the group’s collective efforts. Previous work has shown that using robotics projects in group settings can foster the inclusion of students with learning disabilities.^19–21

The choice of robot typically depends on various factors: for example, if the learning objectives are centered around acquiring software/programming skills, simulation environments or robot model kits might be sufficient.^22,23 Although both physical and simulated robots could be used, simulated robots have less maintenance overhead, are more reliable, and several students can use one copy. On the other hand, physical robots tend to be better received by students; the work by Kurniawan et al. showed that physical robots elicited more positive responses from learners compared to simulated robots.²⁴ If the learning objectives lean towards hardware design and control, a hardware assembly kit, an industrial robot, or an open-source platform might be more suitable. An example is the Blossom robot, which helps students learn by building and customizing their robots.²⁵

Traditionally, mobile robots have been used to teach general-purpose programming concepts because they provide a platform that students can program to move around and interact with the environment.^26,27 In contrast, robotic arms have been used to teach kinematics and dynamics because they have multiple rigid links and actuated joints that serve as good examples for topics that explore the mathematical modeling and control theories that underlie robotic manipulation and movement.^28,29 However, recent trends show mobile robots and robotic arms coupling into hybrid platforms for research and education.^27,30 The level of instruction can also play a role in the choice of robot used, with mobile robots being popular in after-school programs, clubs, and competitions. In contrast, robotic arms are the main focus of university engineering departments. This is because, in the case of tertiary education, industrial-scale robots are needed to prepare students for careers in industry by emphasizing aspects such as hardware, software, and human-machine interfaces.^31,32 This comes with some disadvantages, as industrial-scale robots are expensive to purchase. In addition, there is usually some oversight over their usage due to time-sharing and to prevent damage, which prevents “free play” by students. Some solutions to this include the use of miniature robots and the use of online labs.^33,34 Though these reduce the cost of the setups and allow for more practice time by students, there is a risk that students may not appreciate the true scale of industrial robots and the risks associated with working in close proximity to such robots.

2.2. AR/VR in Education

Advances in hardware and software have made AR/VR head-mounted headsets smaller, untethered, and mass-producible, thus making them readily available to consumers. The prospect of AR/VR headsets being commonplace in most homes is exciting for educators because it opens doors for them to reach more students regardless of geographical location, potentially narrowing down educational gaps by making high-quality course content accessible to a broader audience. VR/AR devices bring with them ready advantages like the ability to completely control the experimental environment and remove all variability, the ability to update experimental protocols on the fly,³⁵ the ability to “damage” components in the environment without actual adverse impact,³⁶ and the ability to create a central repository of the best systems and make them downloadable/implementable by other researchers/users as long as they have the necessary hardware.³⁷ In particular, AR gives researchers the capability to create context-aware environments that blend with the real world, making it straightforward to transfer the skills learned in these environments into the real world.³⁸ In contrast, immersive VR allows educators to completely transport students into a virtual world, meaning that it is possible to recreate hazardous environments in addition to giving educators the capacity to virtually transport students to exotic/complex environments that are not available locally.³⁹

Researchers are exploring the suitability of AR/VR media in tertiary education; they have used these technologies in conjunction with hardware labs to provide immersive and interactive experiences for students, allowing them to visualize and manipulate complex concepts and equipment in a safe and controlled environment while making it possible to partake in frequent and experiential learning activities.^5,6 A growing body of work shows that AR/VR might be sufficient in replacing tangible experiments.^40,41 In contrast, other works do not point to that success.⁴² This mixed outlook brings out the need to further explore the suitability of AR/VR in education, particularly for labs with substantial moving mechanical components, such as industrial robotics. Verner et al. developed a VR-based robotics simulation platform that enabled students to program and control virtual robotic arms. The study results showed significant improvements in spatial perception, mental rotation, visualization tests, and interest in industrial robotics among the participants.⁴³ Similarly, Quintero et al. implemented an AR-based robotic system that allowed students to visualize and program industrial robotic manipulators. They compared the AR programming system with a kinesthetic programming system. The results demonstrated better performance in programming tasks among the users who utilized the AR system, although this could partially be attributed to the AR model being easier to move around.⁴⁴

Intervention strategies implemented during the earlier years of engineering school have proven to be effective in increasing student retention⁴⁵ and success.⁴⁶ In this work, we aim to introduce early-year students to the complex topic of robot programming. This intervention is not only a motivational tool but also a means to study the suitability of AR/VR technology for advanced laboratories. The success of our system in improving student outcomes will give researchers confidence in the potential of using AR/VR technology in education.

2.3. Conceptual Framework: The MUSIC Model of Motivation

The study of motivation is a field rich with a variety of ways to investigate the otherwise unobservable construct.⁴⁷ Some theories seek to characterize the nature of motivation, such as intrinsic or extrinsic⁴⁸; some strive to understand the underlying processes of what motivates an individual, such as in achievement goal theory⁴⁹ or attribution theory⁵⁰; and yet others aim to aggregate beliefs and values that have been strongly correlated to motivated behavior, such as in Eccles’ expectancy-value theory,⁵¹ the MUSIC Model of Motivation,⁵² or the self-efficacy construct.⁵³ In the last category especially, theories and models of motivation are often careful to specify the types of motivation they address. Eccles’ expectancy-value theory, for example, predicts a student’s intentions to persist in a subject and his or her subsequent performance,⁵¹ while the MUSIC Model addresses a student’s motivation to put effort into learning activities.⁵²

In order to maximize the number of factors related to student motivation that our study captured, we selected the MUSIC Model of Motivation. The MUSIC Model asserts that student motivation to learn—i.e., to engage with learning activities—can be explained through five empirically investigable variables, each derived from other theories of motivation in educational psychology literature, described in Table 1. For readers interested in a more comprehensive summary of theories that informed the MUSIC model, see (Brett D. Jones⁵⁴). We believe this model is appropriate for our study, as we designed our intervention as a learning activity that requires active student engagement, which is consistent with the MUSIC Model’s definition of motivation to learn.^52,54 Furthermore, the creator of the MUSIC Model publishes an annually updated, freely available user guide featuring several time-tested interview protocols and validated survey instruments to study motivation using the model.⁵⁵ This user guide allows researchers to study motivation to learn in a way that builds upon—and is consistent with—prior research using the MUSIC Model. Further, the MUSIC Model has been used to study student motivation in engineering contexts, including disciplinary engineering courses,^56,57 engineering student support programs,^58–60 informal engineering learning environments,^61,62 and a senior capstone course.⁶³

3. Implementation

The system implementation and description can be split into three logical parts:

1. Desktop programming interface

2. AR reinforcement system

3. Physical robot reinforcement system

We implemented all the parts in the system so as to minimize the differences between the three sub-systems. It is worth noting that while high-end AR headsets such as those from Varjo come with premium prices and annual licensing fees, the AR technology landscape has rapidly evolved in the past few years. Notably, Meta’s release of high-quality AR headsets at sub-$500 pricing represents a significant shift, making AR solutions much more accessible. In our context, the primary expense lies in software development, as creating a high-fidelity virtual robot simulation requires investment in a VR software developer. However, this cost remains significantly lower than that of a physical industrial robot, making AR a more economically feasible and scalable option in the long term. As AR hardware prices continue to drop, we envision AR-based educational tools becoming more accessible to a wider range of institutions, enhancing their cost-effectiveness and sustainability as a solution for teaching advanced robotics. Furthermore, the development of PC-based digital simulations (like the desktop interface described below) could further improve the accessibility of these simulations.

3.1. Desktop programming interface

The desktop interface was implemented using Unity 3D engine and C# language. We replaced Unity’s in-built physics system with a custom physics engine (AGX Dynamics) to make the simulations more realistic. AGX Dynamics is a high-fidelity modelling suite that allowed us to model and simulate the system with contacts and friction. The user was presented with a log-in screen wherein they entered their participant ID, after which they were led to the tutorial portion of the study. We logged the user ID and stored the user’s program in JSON format for persistence across sessions (if, for example, the system crashed during the activity or if the user wanted to review their program after the study). Figure 2 shows the simple tutorial task where the experimenter walked the user through the steps required to program the robot to move a single block to the highlighted position, while figure 3 shows the more complex task that the user had to complete on their own. In addition, we strived to match the format of the simulation program to the format of the physical UR10 robot program by designing it so that it flowed in a top-down fashion as shown in Figure 1

Figure 1.The physical robot program on the teach pendant(left) vs the desktop/AR program (right)

Figure 2.Task1: The student first observed as an instructor completed a ’simple’ task of programming the robot to pick a block (red) and place it on the highlighted position (blue), then they completed the simple task themselves

Figure 3.Task 2: The student then performed a ’complex’ task on their own. This task required more maneuvering than task1, and the three blocks (red) were initially placed in a variety of positions.

Figure 3 shows the available interfaces and the waypoints. The jogging interface allows the student to move the robot into specific positions, open and close the gripper and pull up the programming interface. This is similar to the controls available when programming a real robot where you can manually move the robot into different positions. The programming interface gives the student the ability to add points and actions to the robot trajectory, edit the trajectory, play back the trajectory and jump between poses of the robot in the trajectory. Finally, the navigation help menu provides an overview of how to navigate in the environment in addition to switching between tasks and resetting the scene and camera views.

3.2. AR reinforcement system

The AR reinforcement system was implemented on a wired Varjo X3 headset, which offers 70 pixels per degree (PPD) – more than the 60 PPD required for a comfortable experience.⁶⁴ The programs created in the desktop interface could be run on a computer connected to the headset and viewed in a AR setting, bringing the robot into the real world with the student. Thus, to ensure consistency, the experimenter created one program that was used by all participants in the AR group, although we should note that it would have been possible to observe the AR robot execute the students’ custom programs. In addition, the program flowed in a top-down fashion which is similar to how the program that ran on the real UR10 robot looked. Furthermore, the programming interface was designed to reflect how an expert would program the real robot by adding waypoints and actions in between the waypoints.

3.3. Physical robot reinforcement system

The researcher programmed a 6 degree of freedom (dof) UR10 industrial robot arm to demonstrate a pick and place task with three aluminum blocks. The UR10 robot uses a scripting language known as URscript which is very similar to Python programming language. The researchers created the program using the teach pendant shown in Figure 5a.

4. Methods and Context

To answer our research questions, we used a qualitative research design consisting of a screening survey, a series of lab-based activities, and individual interviews following the activities. Qualitative methodologies are widely accepted approaches in educational research for studies addressing exploratory research questions about individuals’ experiences.^65–68 Our goal was to identify salient design characteristics of both the desktop app and the reinforcement tools for motivating students. We did so by asking pointed interview questions to participants to understand their experiences interacting with each tool.

The goal of qualitative research is to provide a deep understanding of individual experiences to inform fruitful areas of focus for future study, rather than to establish causal relationships or generalizability. Qualitative studies have low sample sizes due to the intensive amount of time needed to conduct and analyze interviews.^67,69 Accordingly, the results of qualitative research are not intended to yield generalizable insights for a whole population, but rather to dive deep into the experiences of a small number of participants to systematically identify patterns in their experiences. These patterns are described in detail, using direct quotes from participants, so that readers can determine if the described experiences would transfer to a new context outside of the research sample. Accordingly, qualitative research quality is assessed on the basis of the richness of its descriptions for transferability, rather than broad generalizability.^70,71

This approach allowed us to specify which aspects of the research tools mattered for our participants’ motivation and why. It also conferred limitations that can be addressed by further quantitative work. For example, future work could use the published and validated MUSIC Model questionnaires⁵⁵ to assess the effect size of motivational differences between groups and determine the extent to which the differences we observed between groups are generalizable to a larger population of students. Future work could also design other kinds of educational technology to incorporate the salient design aspects we identified and assess whether these design aspects make a difference in the context of other educational environments.

All procedures were approved by the institutional review board (IRB) at the researchers’ institution.

4.1. Sampling

In line with qualitative research best practices, we used a screening survey to purposely sample participants for our study.⁷² The goal of our sampling process was to recruit eight first- and second-year students with minimal prior robotics experience, with at least 40% of participants being women or non-binary in order to balance the gender representation in the sample. However, given the small sample size of genders within each group, it would not be appropriate to discuss gender differences across the groups. The survey asked students for information such as their identified gender, academic year, level and description of prior robotics experience, and, for safety reasons, any known susceptibility to vertigo or epileptic seizures. The survey was sent to all students enrolled in the introduction to engineering courses in the researchers’ College of Engineering. In total, we received 24 responses and selected eight participants to enroll in the study. Half of the participants conducted the desktop activity (described below) followed by observing the real-life robot arm perform an action, and the other half conducted the desktop activity followed by observing the AR robot arm perform the same action. Table 2 shows relevant information about the participants in each treatment group.

Figure 4.A student programming on the desktop robot interface

Figure 5(a) The physical UR10 industrial robot arm with one of the three aluminum blocks (far-right) already placed at the drop off location

Figure 5(b) A student observing the Augmented Reality version of the robot perform the pick and place task on a Varjo XR3 headset.

4.2. Research Activities

Each student spent approximately 90 minutes engaging with study activities. Table 3 provides an overview of the activity schedule across both groups. First, the researcher gave the students a standardized tutorial regarding how to use the desktop robot app, and answered any questions the students had. Second, participants used the app to conduct what we called the “simple task”, which required them to pick up a block with the robot, rotate it to a specific orientation, and place it back down in a specific spot (Figure 4). The purpose of the simple task was to give students a chance to get used to the controls with a relatively easy goal in mind. Students then conducted what we called the “complex task.” This task involved picking up and rotating three blocks (one large, medium, and small) that were scattered around in different orientations, and stacking them as a pyramid in a specific location and orientation. The purpose of the complex task was to challenge students to apply their knowledge of the system to a task that is more involved and builds upon itself.

Once students finished the desktop app programming tasks, the researcher then demonstrated the complex task being completed by a life-sized robot arm. The nature of the robot arm (a physical arm (Figure 5a) vs. a virtual arm (Figure 5b) in an AR environment) depended on the student’s treatment group. The goal of this activity was to observe motivational differences in engaging with a real robot arm versus a life-sized AR recreation of a real robot arm. Finally, after a short break, the researcher collected data via an individual interview with each student.

4.3. Data Collection

We collected data via individual interviews using a semi-structured interview protocol based on an existing MUSIC Model protocol from Brett Jones’ MUSIC assessment guide⁵⁵ and refined with guidance from.⁷³ The full interview protocol is available in Appendix A. The purpose of the interview protocol was to identify what aspects of the research activities students found motivating and why. Interviews lasted approximately 25 minutes and were audio-recorded for transcription. Recordings were transcribed by the parrot.ai artificial intelligence service; researchers retained the original audio files to allow them to verify the accuracy of transcription during analysis.

The desktop programming interface also logged data on participant actions, including keypresses and interface commands like “Undo” or “Reset Pieces.” These data were not the primary focus of this study, but were considered in relation to participant comments. Our results occasionally point to logged data that reinforces what participants discussed.

4.4. Data Analysis

Data analysis was conducted in the MAXQDA qualitative data analysis software, using processes established by.⁶⁷ Particularly, we coded the interview transcripts using a mix of open and a priori codes. As we approached the study with no presumptions about what students would and would not find motivating, we used open codes to identify what aspects of the study experience they described as being motivating. We then used a priori codes consisting of the five MUSIC Model components to classify which aspect of motivation the participant described the activity as affecting (see Table 4 for operational definitions of these codes.)

Once each interview was coded, we used MAXQDA’s visual analysis tools—particularly the Code Matrix Browser and Code Relations Browser—to looks for patterns in how each motivating aspect of the study appearance related to each component of the MUSIC Model, as well as to look for differences in motivation-related experiences between the two treatment groups. Doing so allowed us to answer both of our research questions.

4.5. Research Quality

We took several steps to ensure the credibility, dependability, and transferability of our results, in accordance with guidance from Anfara, Brown, and Mangione⁷¹ and Leydens, Moskal, and Pavelich.⁷⁰ Credibility in qualitative research refers to alignment between the research findings and the social reality under study.⁷⁰ To establish credibility in this paper, we provide direct quotations from participants to allow readers to decide if our interpretations of their statements are appropriate. We also specify to when there was agreement across participants, when participants disagreed on a particular point, and when a point was expressed by only some participants within a group.

Dependability refers to steps taken in qualitative research to ensure that data collection and analysis procedures are applied consistently throughout the study.⁷⁰ During data collection, we ensured consistency by (1) preparing a detailed study protocol guide such that each participant had the same amount of time available to engage with each study activity (Table 3); and (2) preparing a semi-structured interview protocol and yield responses that could be compared across participants (Appendix A). We ensured dependability during data analysis by (1) conducting an intercoder reliability process to ensure refine the definitions of our MUSIC Model codes to match how they were applied (Table 4); and (2) the second author engaging in regular peer examination of codes applied by the first author to ensure the codes were applied consistently.

Finally, transferability refers to the ability of readers to determine if the results of the qualitative research in the researchers’ context would reasonably apply to the reader’s context.⁷⁰ We established transferability in this paper by describing the applications being studied, and the laboratory context in which they were studied, in detail (section 3 and subsection 4.2, respectively).

4.6. Limitations & Appropriate Interpretations

The primary limitation of this work is the sample size associated with the study. Qualitative research intentionally uses small, purposefully selected samples in order to allow researchers to explore each participant’s experiences in-depth,⁶⁷ and there is no generally applicable rule about appropriate qualitative research sample sizes. Rather, qualitative research often proceeds in waves of iterative data collection and analysis until the researchers achieve “saturation”: the point at which introducing new data does not change the study’s results.^74,75 However, achieving saturation is a time-intensive process, and like all studies, the design of our research was bounded by the resources available to us.⁷⁶ Particularly, this study was conducted as the main component of a semester-long, self-contained educational research training experience for this paper’s first author. We scoped the study’s design and intended impacts based on the time constraints of this arrangement.

Accordingly, we encourage readers to interpret the results of this study as a preliminary window into what factors mattered to motivate students to engage with this novel introduction to robotics programming. We acknowledge that our findings are not generalizable to larger populations of students, but we also believe the patterns we saw within our sample have salient implications for the use of this type of intervention for early post-secondary robotics education. In keeping with qualitative research best practices, we include a detailed description of our intervention and many direct quotes from our participants so that readers can decide whether or not our results could transfer to their own robotics education contexts.^70,77

5. Results

To answer the two research questions introduced in section 1, we focused on trends in motivation across both the AR and physical robot arm groups, as well as differences between the two. Based on the interviews, we found three overall themes in what affected students’ motivation: (1) the design of the desktop programming interface; (2) the structure of activities around the desktop programming interface; and (3) the nature of the reinforcement system (AR vs. physical robot).

5.1. Design of the desktop programming interface

Participants across both groups expressed admiration toward the interface’s design. For some, it was because the manner of interaction mirrored some commercial software they had previously used. For others, it was because the interface gave them comfort and mobility in the environment as they carried out the block-stacking tasks. These observations can be attributed to the interface’s functionality. However, some users mentioned that they appreciated how closely the interaction in the exercise scene matched real-world physics and how it did not feel scripted or pre-programmed. Also, some liked the interface’s design (describing it with words such as “awesome” and “cool”), or described feeling like they were playing a game. These observations can be linked to the aesthetics of the interface, the realistic physics engine used to simulate motion, and the game-like design of the scenarios. As one participant said:

I would say [I was] motivated, I mean I wasn’t bored or anything like that. It was presented in a game format […] I was just moving the little blocks around, so that was fun. (202)

Overall, the comments about the interface’s design strongly corresponded with the interest component of the MUSIC model.

5.2. Structure of activities around the desktop programming interface

All participants reported feeling very motivated after completing the block-stacking assignment effectively by aligning the end position of their blocks with the exercise pyramid. The participants appreciated having enough time to do the task multiple times until they were satisfied that their program could appropriately stack the blocks. In addition, other participants appreciated being able to play with the tool and receive a tutorial from the experimenter before having to program the robot independently. Finally, some participants mentioned that the independent programming exercise at the end motivated them because it allowed them to practice and showcase the skills they had learned. Below are some excerpts from students appreciating being able to get it right:

[…] And so then when I got [the solution] wrong, you asked me if I wanted to retry it, [and] I wanted to get it right. Like I just, I wanted it to be perfect. So that was exciting. (103)

What really made me feel successful was playing back the waypoints that I set up in here and out there and seeing it all come together and work, that was really gratifying. (201)

Overall, the application of the tool corresponded primarily to the success component of the MUSIC model, and secondarily to its empowerment component. Students’ perceptions of their ability to succeed were also aligned with their actual success to completing the research tasks. Figure 6 shows that different students needed different amounts of experimentation while using the program (“Reset Pieces” was the command each student used when attempting a new solution). Some students needed nearly ten attempts, while others needed only one or two. Regardless, every student eventually solved the problem they were provided, suggesting that the program’s design supported success for students with diverse incoming prior knowledge and preparedness.

Figure 6.Number of times each participant reset block positions, as a proxy for the number of solution attempts. Some students needed more attempts than others, but all successfully completed the task.

5.3. Reinforcement: AR Robot vs Real robot

The groups differed in their motivation-related perceptions of the reinforcement activity. The participants in the physical robot group reported being able to link their efforts in the desktop tool to the actions of the real robot. Some of them were cognizant of the speed and accuracy afforded by the robot arm and were able to associate this with ways in which the robot could be used to solve problems in the real world. Two students in the physical robot group expressed this sentiment well:

My key takeaway is I still think robotics is very fascinating. I think the things that you can do with it are very impressive and the fact that you can take something virtual and make it so efficiently and precisely done in the real world is a very cool concept to me. (204)

I think it’s interesting to see because we all know robots can replace humans, but it’s interesting to see how it can be used to cut down time. For example, in research, instead of you doing things manually you can get a robot to do things like pick up some things and do some things [you’re] doing in a process. So you don’t have to do it like 100 times. So it’s just interesting to see that. (203)

Conversely, participants in the AR robot group had less to say about the real-world applications of robotics, but instead reported the experience as being highly novel to them. For example, one participant discussed that they enjoyed seeing the exercise they had just completed in the programming interface “come to life”:

It was nice. I’ve never used VR so it was cool seeing it. It was really interesting like seeing it interact with the blocks like not just on a computer screen. […] and knowing I programmed it in that sequence was cool, seeing it come to life. (102)

Some participants in the AR robot group expressed some difficulty tying the experience with real world robots. For example, one participant commented on their perception that the AR robot arm was an oversimplified representation of a physical robot:

Well programming a real [robot arm] would take much more work and be much more difficult. But I think if given enough time, I could learn how to do that. Since everything I learnt was at the beginner level, I’ll definitely remember that in the future if I ever do that. (104)

Based on the student comment above, one might expect that participants reinforced with the physical robot would express similar reservations about the programming task being oversimplified compared to a real-life robot. On the contrary, some participants in the physical robot group expressed that seeing a real-world robot perform the same task they just completed in the programming interface helped them see how robot programming works in the real world. As one participant commented:

It mainly just showed me that whatever I can do virtually I can do physically too. So when I watched the robot arm do pretty much the same thing I had done in that program, it made me feel confident that if I wanted to do something with the robot arm, if I wanted to seek out a career or something in robotics, then that is the kind of work that I would probably be doing. Not necessarily all the work, just working in the program might obviously be doing programming stuff too, but at least that aspect I have confidence in myself now. (204)

Overall, students in the AR robot arm group reported the interest portion of the MUSIC model as a major driver of their motivation, with novelty being a big driver of this observation. By contrast, students in the physical robot arm group commented more on the Usefulness portion of the MUSIC Model, being able to more easily connect the experience to real-world robotics. We observed similar levels of scoring in the success and empowerment components of the MUSIC model in both reinforcement groups.

6. Discussion

The implementation of a desktop robot programming interface for active learning coupled with observing a programmed life-size robot arm was very encouraging. Results indicated the presence of strong motivating factors that cut across the board from the 2D interface to the 3D environments. In addition, the similarities in motivating factors in both the AR and physical robots activities point towards the possibility using either as the reinforcement tool. We view this outcome as encouraging for the accessibility of robotics education, as a headset through which to experience the AR robot arm is several thousand dollars cheaper than purchasing a real robot arm for educational purposes (and could be adapted to mobile devices for even cheaper.) However, there were interesting differences between the two that warrant further investigation before we get to that stage. In particular, the two modalities of seeing a life-size robot arm in action were motivating in different ways. When the AR robot was used for the observation part, students found it highly novel, in fact all the participants in the AR group commented on its novelty, compared to only 1 of 4 in the physical robot group. The students that observed the physical robot for the reinforcement part were able to readily link the experience with usefulness and they were able to draw parallels with real world applications. In general, motivation literature suggests that usefulness is a more potent motivational factor compared to novelty-driven situational interest.^52,78 Accordingly, if the AR robot arm were to be used to introduce students to a life-size robot arm in action, instructors may wish to introduce other real-life elements of robotics (smaller real-life robots students can tinker with, or examples of what real-life robot programming looks like) to improve student perceptions of the usefulness of the learning experience.

The desktop robot played a foundational role in the study because the students used it for active learning before they observed a life-size robot arm. Overall, students associated the desktop tool with the interest part of motivation in the MUSIC model; particularly, the students found the interactivity and game-like design of the interface to be a major motivating factor. In addition, the students reported feeling the greatest amount of success when they were able to program the desktop robot to perform the stacking tasks. Motivation literature—especially around game-based learning—has highlighted the importance of low-stakes play and early experiences of success in building student motivation and self-efficacy,^52,53,79 emphasizing the importance not only the design of the desktop app, but also how it is used. In implementing the desktop app into a classroom, it would be vital from a motivation perspective to do so in a way that allows students the time necessary to play, experiment, and successfully complete the tasks assigned to them.

The desktop robot was designed as a readily available learning tool that introduces the major concepts of technical industrial robot arm programming, that is, creating a trajectory, evaluating the real-time performance of the robot arm across a variety of metrics (like accuracy and speed), and obstacle avoidance. Although a 2D interface and lacking the depth associated with the real world, the desktop version serves the purpose of introducing students to the concepts listed above as a precursor to interaction with a life-size robot arm in the form of an industrial robot arm in physical form or in AR. The life-size robot arm allows the user to apply 3D visual cues on the robot trajectory in the real world and reinforces the concepts picked up in the 2D robot exercise. In other words, the 2D desktop robot serves to lay a programming foundation for the student while the life-size robot arm (whether AR or physical) reinforces these concepts in a workforce-like setting. Conceptually, it is akin to a flipped classroom model where the learner interacts with the course material on their own time (programming on the desktop version in our case), and filling in their knowledge gaps at their own pace before practicing the concepts with an instructor present. Interested researchers and instructors can download the code for the desktop robot, including the infrastructure for running a study, in our open source github repository: https://github.com/velaboratory/desktop_robot_uga.git. We hope to encourage interested instructors to make use of it as a way to introduce robot programming to engineering students. The corresponding authors are available to assist in the event of issues during implementation of the code.

7. Conclusion

In conclusion, the desktop robot interface, though lacking in some realism aspects, was found to be a useful tool in motivating students. However, we need to be cognizant of the fact that the design of such a tool and its implementation in a learning setting plays a role in motivating students. The AR and physical robots were found to be motivating in different aspects: the physical robot was stronger than the AR robot in the usefulness category while the AR robot was stronger in the interest category. Overall, the motivating factors were similar in both life-size robots and they might be used as drop-in replacements for each other, or in conjunction with one another.
A limitation of this work is that it was a proof of concept and was not used to teach students as part of an engineering course, therefore, it lacks ecological validity. Furthermore, given the small sample size of genders within each group, we could not discuss gender differences across the groups. In order for the tool to be integrated into an engineering course, it is essential to consider accessibility and inclusivity by incorporating Universal Design for Learning (UDL) principles. While the tool was designed to provide greater access to a realistic robot programming learning environment, it could be designed to more equitably reach a wider range of students through features such as the use of hand-held tablets for AR to increase accessibility or the use of haptic, sound and visual effects in the application. Further, UDL principles can be implemented in physical learning environments through universally designed structures and accessible pathways, such as wheelchair-accessible entrances. Sufficient space for interaction, such as a room-scale play area, should be provided, and VR hardware must be adaptable to meet diverse user needs, including appropriate height adjustments. In virtual environments, developers should eliminate interaction mechanisms that rely on properties inaccessible to certain populations. For instance, interactions should not depend solely on color differentiation or require fine motor skills. Enhanced locomotion options, such as snap turning and teleportation, can further facilitate navigation for learners with varying needs. Curricula should also align with UDL principles by supporting learners’ diverse recognition, strategic, and affective networks. This can be achieved by including multiple examples, offering opportunities for practice with scaffolding, and providing adjustable levels of challenge to meet the needs of all learners.⁸⁰

Future work is projected to include a fully immersive robot-programming interface which, when combined with automated tutorials and assessments, could function as an independent, widely distributable training platform, greatly expanding accessibility. However, challenges in such a system include the need for a robust multi-user environment, particularly if instructional guidance is required in real-time. Such environments demand carefully designed tools that balance student autonomy with necessary instructor controls. Additionally, to maximize the pedagogical value, the developers must carefully decide how closely to replicate the physical robot’s interface, aiming to retain the aspects most essential for learning while simplifying where feasible.⁸¹

Appendices

A. Interview Protocol

Thank you for taking part in the study and I hope you enjoyed programming the robot. This is the last part of the study, and we will be interviewing for around 25 minutes to get your thoughts on your experience during the robot control activities. You have received a consent form to sign, which indicates your consent to this interview. The interview will be recorded.

1. (Icebreaker question #1) Tell me a little bit about what you’d like to do once you graduate.

2. (Icebreaker question #2) We would be interested in knowing what inspired you to participate in this research. What about it appealed to you?

3. (How motivated) First, how motivated were you to engage with the activities we did today?

4. (Why motivated) Second—and this is the main question—what are some things about the activities we did today that helped motivate (or demotivate) you to stay engaged?

   1. How did you feel about experiencing the real-world robot arm example (in-person/via virtual reality)? How motivating was that experience in terms of wanting to learn more?

(The following questions are intended as follow-up questions if students do not touch on them when answering Question 4. These questions were adapted from.⁵⁵)

5. (Usefulness setup) What were your key takeaways from the activities?

6. (Usefulness) What did you find useful about the activities?

7. (Success) What made you feel successful during the activities?

   1. How did the activities affect your perception that you could be successful engaging with robotics in the “real world”?

      1. How did seeing the life-size robot arm in action help with this perception?

8. (Interest) What did you find enjoyable or interesting about the activities?

   1. How did the activity(s) relate to your personal interests?

9. (eMpowerment) What meaningful choices did you have when completing the activities?

(The following questions are wrap-up questions to be asked in the last 5-10 minutes of the interviews.)

10. What could we have added or done differently in these activities to help motivate you more?

11. (Back pocket question) Okay, you have been very helpful. Any other thoughts or feelings you might share with us to help us understand your experience with programming the robot and how it affected you?

12. Do you have any questions for me?