Introduction

Performing maintenance on equipment and assets can be a lengthy and complicated task, especially when the operation and maintenance manual is confusing to read and hands on training is limited. This augmented reality electric motor maintenance application equips technicians with an on the job training tool, enhancing accuracy through real-time instructions, visual aids, and automated logging processes.

The only design revisions since the previous devlog are those resulting from the usability testing. These changes are discussed in further sections.

Technical Development

Model Target Type

The application was developed using Unity and the Vuforia Engine. Model targets are utilised as the tracking method, allowing them to recognise the shape of the motor undergoing maintenance.

Scripts

CenterObjectChecker – Checks which object is currently in the centre of the screen, changing the objects material and displaying its stored name and description on a text object. This script is used during the general inspection task to highlight and display the information of each motor part.

ObjectInfo – Stores the name and description of an object in Serialized fields. Used alongside CenterObjectChecker to retrieve and display the information of each motor part.

PartIssueLogger – Used to store the issues recorded for a particular part by using a dictionary that holds a given part and a list of its issues. The script also displays them for review in a different text object within the Logged Issues section under Task Summary.

QuitButton – Used in the final panel when the user “submits their report”, closing the application after the submit button is pressed.

SceneChange – Used to change between the Menu and MotorScene when the user clicks the proceed button.

TaskComplete – Keeps track of which tasks have been completed and updates text within the “Task Summary” section, originally automatically notified the user when the report is ready to submit however this was changed to a checkbox check for cases when all tasks did not need to be completed or tasks needed to be redone for clarity.

TextToText – Updates the text in the “Task Summary” section that handles the motor and user info first inputted when starting maintenance on the motor.

ToggleVisibility – Used to handle transitions between different panels on button presses by hiding the buttons parent object and activating a target object, in most cases another panel.

Tasks/Scenes

Initial menu scene – Gives general instructions on how to use AR and the application, uses a panel to frame text and a button to prompt user to switch to the Maintenance scene.

Maintenance scene – Handles all of the maintenance and testing tasks/ information summaries.

Info Sub Menu – The first panel the user sees in the scene after scanning the motor, gives information on the motor type and prompts the user to enter the given motor number, the users name, the date and the location the testing is taking place in. The User can then use the “Enter” button to submit this information and the “Go to Main Menu” button to switch to the main task selection panel. The info Menu can also be accessed from the main task panel to adjust any details if required.

Main Menu, Maintenance Menu and Testing Menu – Uses Buttons to allow users to switch between different tasks.

Maintenance tasks

• Cleaning – Shows users how to use a brush to clean a motor, playing a custom animation when the demonstration button is pressed.
• Basic inspection – Prompts users to inspect the motor by comparing the virtual motor and its maintenance information with the real motor.  After finishing the inspection, the user is then prompted to select any parts that showed issues using the dropdown menu and describe the issue in a text box before pressing the enter button to submit the issue.
• Lubrication – Shows users how to properly lubricate a motor by playing a custom animation.

Testing

• Electrical – Shows users how to properly test the current of a motor using a multimeter, with a custom animation being used to demonstrate.
• Sound – Shows users what a good and bad sounding motor sounds like, prompting users to redo maintenance tasks if an incorrect noise is occurring.

Task Summary Menu – Lists a summary of all the tasks to be completed by the user with a checkbox next to each task which can be checked to indicate completion, checking all boxes indicates that the final report is ready to be submitted.

Issues log Menu – Lists all the parts that the user indicated had issues alongside a list of any given issues.

Final screen – Notifies user that they have finished all of their tasks and prompts user to click a button to submit a report which also quits the application (note this function would hypothetically send a report to a company database and is not able to be fully implemented).

3D Content

Usability Testing

Usability testing was performed to assess the overall user experience and effectiveness of the application, aiming to uncover any design issues or missing elements.

Design and Plan

For the application to be successful, it needs to be user friendly, effective, and time efficient for technicians to use. In order to ensure that the application meets these objectives, success requirements and usability test tasks were developed for the application. Success requirements are the essential tasks and functions that users must be able to perform with the application for it to be deemed successful. The success requirements for this application have been identified as follows:

• R1 – Motor detection
• R2 – Enter motor information
• R3 – Perform a maintenance task
• R4 – Perform a testing task
• R5 – View logging information

Below are the usability test tasks aligned with the application’s success requirements:

• T1 – You need to perform maintenance on you motor. Startup the application aligning the motor in the camera frame and enter the motors specifications.
• T2 – The motor requires cleaning. Perform this task.
• T3 – The motor requires an inspection. Perform this task. You also notice the rotor is showing signs of wear, note this during the inspection process.
• T4 – The motor requires lubrication. Perform this task.
• T5 – Your manager has asked you to test the electrical components of the motor. Perform this task.
• T6 – You notice the motor does not sound right. Investigate this further by performing a test.
• T7 – Check all tasks that have been completed.

Upon generating the success requirements and usability test tasks for the application, a usability task matrix was created that maps the two together.

Table 1: Usability Task Matrix

For each task, it was decided to record the time taken, whether it was completed successfully, any errors that occurred, and any participant or experiment conductor comments. A post testing questionnaire was also created for participants to fill out after completing all tasks. The questions were designed to focus on the clarity of instructions, application intuitiveness and navigation, and the overall usability within the domain of the application. The questionnaire uses the Likert scale.

Recruitment

Ideally, the usability testing for this application would have included actual technicians, who are the primary target audience. However, it was not feasible to source the technicians and physical motors required for this prototype testing. This may introduce a potential source of bias in the usability testing process because the data collected does not directly represent the experiences and needs of technicians who would typically use the application. Instead, the usability testing was conducted with friends and family members, which is a convenience sample. Participants in this sample do not possess the same level of expertise or familiarity with maintenance procedures as professional technicians. In an attempt to mitigate this bias, participants were asked to take a role-playing approach, effectively putting themselves in the shoes of the target audience.

Considerations related to age, gender, and prior AR experience were addressed by ensuring a diverse range of participants across these demographics. Among the five participants, three were female, two were older than 40, and two had no prior experience with AR, with one having only limited exposure to it.

Protocol of the Testing

The tests were carried out in a home environment in a one-on-one setting during the day. While it would have been preferable to have the same experiment conductor, there were only two different conductors which was unlikely to cause significant issues. During the introduction, participants were instructed to role play as technicians and were provided with an explanation of the role and application itself. Each participant was given one task at a time, with the experimenter recording the time taken, noting any errors, and collecting any comments. After completing all the tasks, the participants were asked to fill out the post testing questionnaire.

A limitation of this usability testing was the unavailability of a physical motor for testing. Consequently, participants were required to point their device at an image of the motor for testing instead. This also meant that the participants were asked to imagine that they were following the steps of the maintenance tasks as a physical motor was not available for hands on interactions.

Report of the Findings

All raw data for each participant can be found in the appendix. The summarised results of the usability testing for each task are presented in the table below.

Table 2: Usability Testing Results

The summary of the post testing questionnaire results is presented in the table below. 

 Table 3: Post Questionnaire Results

Analyses of the Findings

Table 2 shows a 100% success rate for all tasks except Task 1, where one older participant faced challenges in understanding AR, possibly due to age related factors. There were no major errors recording other than this and one participant struggling to locate the area for inputting information on motor parts wear. This suggests the application meets the success requirements, although more comprehensive instructions may be necessary for users who are unfamiliar with AR. However, participant comments and those made by experiment conductors indicate that some changes are needed in certain aspects of the application. The average completion times for each task varied, with Task 1 taking the longest with 61 seconds and Task 7 the shortest with 14 seconds. Tasks 2 through 6 showed similar completion times, ranging from 30 to 40 seconds. Task 1's longer duration can be attributed to the initial steps of aligning the motor and inputting motor information, while Task 7's short duration is understandable since it only involved viewing the logs. These results are challenging to quantify when assessing the time efficiency goal of the application. The participants had to role play the tasks rather than directly performing the maintenance, which reduces the significance of the recorded times. Further testing with actual engagement in the tasks would be necessary to accurately measure the application's time efficiency.

Examining the post testing questionnaire results in Table 3, it is evident that 60% of the participants strongly agreed that they could see the application being useful in its domain, while 20% simply agreed. This indicates the potential for the application to be a valuable tool for motor maintenance. However, the 20% neutral response suggests that more data is required to fully support this claim. Referring back to the goal of ensuring user friendliness, the feedback was largely positive, indicating the application's success in this regard. Most participants either agreed or strongly agreed that the interface was intuitive and that the navigation within the application was seamless. However, the 20% neutral response suggests that some refinements may be required to further improve the interface intuitiveness. Another goal of the application was to be effective in its use case, however, due to the limitation of not having a physical motor and technicians during the testing, it was not possible to fully assess this. 80% of participants either agreed or strongly agreed that the instructions were easy to follow, with 20% remaining neutral. However, the quality of this assessment is limited as participants were unable to physically perform the instructions. Further usability testing with an actual motor, preferably involving technicians as participants, would be necessary to fully evaluate the goals of the application.

Addressing the Results of the Usability Testing

The usability testing generated valuable feedback from participants that prompted positive changes to the application. To address the issues encountered in the first task, clearer instructions were implemented at the beginning to assist users who are less familiar with AR. In future versions, these instructions could appear when attempting to align the motor for more clarity. Also throughout the first task, some participants noted that the motor information activity was confusing and not intuitive to enter initially. To address this, a prompt was added after the motor had been aligned, and the layout was reorganised for clarity. For the maintenance and testing tasks, several participants felt that the demonstration animations were too fast, and as a result the animations were slowed down. In future versions, incorporating a slider option to control the speed of the demonstration could provide users with more flexibility and control.

On the third task when performing a basic inspection, several participants noticed that there was overlapping of text in the drop down menu. This issue was fixed accordingly. Some participants also indicated that the section to add any issues found during this task was too small and difficult to understand, which prompted adjustments to improve this. Additionally, a participant suggested highlighting the part that was being inspected during this task to improve clarity, leading to the implementation of this feature. During the electrical testing task, a participant commented that the multimeter model was too large compared to the motor. However, this issue was due to needing to use an image instead of an actual motor, thus when a physical motor is used this will not be an issue. Another participant found it challenging to see the multimeter connections clearly. This issue can also be addressed when an actual motor is used as the app can prompt the user to rotate the camera or motor to see the area more clearly.

Finally, there were some comments regarding the logging feature that resulted in several changes to the application. One participant's feedback regarding the interface design led to the replacement of text with checkboxes to indicate completed tasks. Additionally, the interface underwent some clean up for a more user-friendly appearance by increasing text size. Another participant commented that the purpose of the "Logs" section was not clear from its name, prompting an update to the section's labelling for increased clarity to “Task Summary” Lastly, a feature to prompt the user to submit a hypothetical report upon completion of all tasks was added based on a suggestion from a participant.

References

3D Models:

[1] Motor DC animation by davidlecardinal via sketchfab https://sketchfab.com/3d-models/motor-dc-animation-11bb9b2605d84a57aadc04d8654096da

[2] Low Poly Paint Brush by TheoClarke via sketchfab https://sketchfab.com/3d-models/low-poly-paint-brush-54f105b0be154bbbae49d1d3aae2c6e1

[3] Multi-meter Final Model by bsuvderdene8 via sketchfab https://sketchfab.com/3d-models/multi-meter-final-model-a7919b0603df4a7e9e9e9e3250ab6322

[4] Flathead Screwdriver by AleixoAlonso via sketchfab https://sketchfab.com/3d-models/flathead-screwdriver-240935915a744929914369cc12816ab2

Sounds:

Electric Motor shaft lubrication by furulevi https://www.youtube.com/watch?v=L_0SKz8aHRk&list=LL&index=6

Bleep by Pixabay https://pixabay.com/sound-effects/bleep-41488/

Videos:

12v Motor Rewinding, RPM & Power Increase 555 Motor Recoil by SAYAN DALAL 12v Motor Rewinding, RPM & Power Increase 555 Motor Recoil

How to upgrade and repair your DC motor by using some lubricant by Rounak Roy How to upgrade and repair your DC motor by using some lubricant

Code:

ChatGPT conversations:

-        https://chat.openai.com/share/5f849d81-eb7a-4448-8342-bf474b23e981

-        https://chat.openai.com/share/befba857-677d-4bcf-b116-b86acaf87acd

Appendix: Usability Testing Raw Data

Table 4: Participant 1 data

Table 5: Participant 2 data

Table 6: Participant 3 data

Table 7: Participant 4 data

Table 8: Participant 5 data