 |
| “This is a more intuitive way to command the machine,” says Hermano Krebs. Krebs sees the interface as a faster way to train excavator operators, as well as a new way to physically operate the machines, both on-site and remotely. Photo Credit:: Courtesy of the researchers (CC BY-NC-ND 3.0) |
Scientific Frontline: Extended "At a Glance" Summary: Excavator Training Interface
The Core Concept: MIT engineers, in collaboration with Sumitomo Heavy Industries, have designed a new, intuitive training interface for excavator operators that uses a miniature mechanical arm to mirror real-world movements, replacing traditional joystick controls.
Key Distinction/Mechanism: Unlike standard joysticks that require operators to build a complex "mental map" to coordinate movements, the "World-Space Interface" (WSI) allows trainees to physically mime the actions of an excavator's arm and bucket. A computer translates these natural arm and hand movements into actions for a digital excavator in a virtual simulator.
Origin/History: The collaboration between MIT and Sumitomo Heavy Industries began in 2018. It was prompted by a rapidly aging workforce of heavy machinery operators in Japan and the need for a faster training method to replace them.
Major Frameworks/Components:
- Miniature Mechanical Arm: A physical controller that trainees grasp and move, mimicking the desired actions of the excavator.
- Immersive Virtual Simulator: A six-screen digital environment that projects an excavator mirroring the trainee's movements across 15 realistic excavation scenarios (e.g., construction sites, mining areas).
- World-Space Interface (WSI): The combined platform of the mechanical arm and software that translates the user's natural movements into the "world-space" (the environment outside the cab).
- Haptics (In Development): Future iterations aim to include force feedback in the physical arm, allowing the operator to "feel" the resistance or weight of objects being manipulated in the simulation.
Branch of Science: Mechanical Engineering, Human-Computer Interaction, Robotics.
Future Application: The WSI could be used for rapid training of novice operators and eventually be integrated directly into excavator cabs or used for remote teleoperation in hazardous environments.
Why It Matters: Traditional excavator training can take years to master due to the non-intuitive nature of joystick controls. This new system allows novices to perform at the level of experts almost immediately, significantly reducing training time and addressing critical labor shortages in the construction industry.
“This is a more intuitive way to command the machine,” says Hermano Krebs, principal research scientist in MIT’s Department of Mechanical Engineering. “With this new interface, we can eliminate a lot of the mental maps that an operator would need to build in order to operate an excavator.”
Krebs sees the interface as a faster way to train excavator operators, as well as a new way to physically operate the machines, both on-site and remotely.
“Instead of having joysticks, you might have this miniature arm on the side, where the operator would place their own arm, kind of like an exoskeleton, which would allow them to operate the excavator in the cab,” Krebs says. “If work has to be done in a difficult or unsafe environment, you could have an operator sitting off-site in a trailer and using this arm to remotely teleoperate the excavator.”
A Machine Mimic
At MIT, Krebs’s group works on human-robot interactions, with a longtime focus on physical rehabilitation. Through this work, the team has accumulated knowledge about the ways in which humans control their limbs and how they can most intuitively interact with machines.
In 2018, Krebs struck up a collaboration with researchers at Sumitomo Heavy Industries, who were looking for a faster way to train excavator operators. They noted that in Japan, the population of heavy machinery operators is aging rapidly; training their replacements takes time.
Operators typically learn by driving actual excavators on a controlled driving course. As they operate the machine, novices must learn to relate the actions of the excavator’s joysticks with the movements of the arm, bucket, and cab. Coordinating these actions to carry out actual tasks adds another level of complexity that can take months to years to master.
The team reasoned that if they could eliminate the need for this mental map, they might significantly shorten the training process. To do so, they looked for a more natural way to control the machine as an alternative to the traditional joysticks. They soon landed on the mechanical arm design, reasoning that the physical resemblance to the digger’s own arm and bucket could enable operators to mimic and control the excavator’s movements directly, without much mental translation.
Over the next few years, the researchers worked to build the mechanical arm, along with the software to pair its movements with a virtual simulation of an excavator. The combination of the mechanical arm and the virtual simulator constitutes a new training and control platform for digger operators, which the team has named the “World-Space Interface.”
“‘World-space’ refers to everything in the world that is outside of yourself, or in this case, outside of the excavator’s cab,” Krebs explains. “Normally, operators have to build a mental map of how to manipulate things in the world-space. But now, we can just mime picking up rocks or dirt, and the computer will do that translation to the world-space for us.”
Construction on Day One
For their new study, the team ran training experiments with volunteers who used the World-Space Interface (WSI) as well as a more traditional, joystick-based excavator simulator. The researchers developed virtual simulations of fifteen realistic excavation environments, including construction sites, highways, forest roads, riverbanks, mining areas, and urban and rural settings. Each virtual environment was associated with various excavation tasks, such as scooping and dumping sand or gravel, digging and grading trenches, clearing debris from roads, removing tree branches from water edges, and breaking up rocks.
The team designed the experiment to resemble the tasks that an operator typically performs during a weeklong excavator driving course. For one hour each day for seven days, volunteers—both expert and novice—operated the WSI and the joystick counterpart, training on tasks with increasing difficulty.
The researchers then compared the volunteers’ performance before and after the training period. For the joystick simulator, they found that novices were consistently worse than experts, though they did improve over the training period. In comparison, the team found that with the new World-Space Interface, novices were just as good as experts from the start.
“In this case, joysticks are a nonintuitive way to control and coordinate the machine,” says study coauthor and MIT postdoc Joao Buzzatto. “This is the first interface that does not require me to command the excavator with joysticks.”
The team is now working to add haptics, or feeling, to the WSI’s physical arm. The idea is that as an operator uses the arm to mime an action such as picking up a pile of rocks, the arm will generate a force in response, as if the operator can feel the heaviness of the rocks, as confirmation that the excavator is indeed picking them up.
“Haptics would make this an even more intuitive system,” says coauthor and visiting engineer Solmon Jeong.
The team says the new training interface can be a more natural alternative to excavator simulators that the construction industry is currently exploring. Companies such as Caterpillar, Hyundai, and Komatsu are developing virtual simulators, both to help train operators before they go on-site and to one day remotely control excavators from a distance. However, these simulators are largely based on traditional joystick controllers that still take time to learn.
If the team’s new arm-and-bucket controller were incorporated as an appendage in an excavator cab or in a virtual, teleoperational simulator, the researchers envision that even first-time operators could get to work from day one.
Funding: This research was supported, in part, by Sumitomo Heavy Industries.
Published in journal: Journal of Computing and Civil Engineering
Authors: H. Ishida, M. Alencastre-Miranda, T. Hoshino, A. Martinez Guerra, S. Jeong, D. Suzuki, J. Buzzatto, E. Bamani Beeri, D. Usui, S. Tsukahara, and H. I. Krebs
Source/Credit: Massachusetts Institute of Technology | Jennifer Chu
Edited by: Scientific Frontline
Reference Number: eng082026_01