Can Supervised Autonomy Fix the Operator Shortage?

Supervised autonomy takes operators out of the seat and puts them in an office or trailer running what looks like a deluxe simulator.

Unlike full autonomy, in supervised autonomy a human being remains in charge of the machine even if that machine is miles—or states—away.

Mark Piotto, VP of sales for Teleo, an innovator in autonomous technology, thinks it’s a way to solve today’s labor shortage, but there will be some resistance, much like there was for GPS and machine control.

Adoption of autonomy will take time

“I’ve met hundreds, if not thousands, of equipment managers that had this attitude: Technology is important [to them], but they’re so busy fighting a battle doing their day-to-day activities they don’t want to think of change," he said.

Piotto spent 20 years of his career selling machine control systems.

“I can’t tell you how many offices I’ve been kicked out of trying to sell a $100,000 box when you know that they just don’t believe in the technology,” he said. “The evolution of technology takes a while; I put the timeline for GPS at 30 years to take the business from a four-person [surveying crew] to a one-person crew.

“If we don’t take a chance on new technology, we’re never going to evolve,” Piotto said. “Now it’s hard to buy a machine without grade control on a dozer or motor grader today.

“The problem I was solving for the last 20 years was the skilled operator, or lack thereof,” Piotto said. “Today’s challenge is finding an operator, period. Somebody who wants to drive two hours to get to then work in a harsh environment. These can be very harsh environments, hazardous physically and stressful. You’re sitting in the cab of a machine for 8 or 10 hours a day. Today people can take pictures of their food on Instagram and make more money.”

Piotto asked, “So why would anyone want to work in the dirt?”

Read also: The Case for Remote Control

He cited statistics that say 93% of contractors are looking for people. Of those 93, 91% are having a “real difficult” time filling seats. He said it’s only going to get worse.

“By 2031, we’re going to lose 41% of our labor force—but wait, we have a $1 trillion infrastructure bill and we’re going to have to do more with less people,” Piotto said.

Autonomy, which is alive and well in mining, has not reached the mainstream on construction sites. But Piotto believes that day is coming.

“Tesla has supervised autonomy, if you will. You can put it in autopilot and you’re supposed to pay attention as you’re driving and if you don’t, it’s supposed to stop," he said. "But if it doesn’t, we have an accident. That really is a version of supervised autonomy, where there’s a human in the loop helping the car learn how to operate in certain scenarios.

"I believe that supervised autonomy is the road to allowing contractors to do more with the operators they have, particularly when one operator can operate multiple machines at the same time,” Piotto said.

What’s more is that supervised autonomy may attract a fresh generation to the industry.

“We really don’t want to get rid of people. We feel that if somebody can sit in the comfort of an office and operate a machine remotely as we do everyday today, we can attract the younger generation, the single parent that can’t drive two hours to get to a job site, and wounded veterans that can’t get onto a machine," Piotto says.

He gave an example where an operator could sit in Texas and run a machine in Finland and two in California at the same time.

How Teleo remote control works

The operator controls the complex tasks. In a loader, the complex tasks are picking up material, making sure you’ve got a healthy load, and dumping the material. But the traveling between the pile and the truck is being done autonomously. Everything is on the large real-time monitor.

“Our operators are here to see and feel the vibration, including slightly haptic feedback in their chair," Piotto says. "You can see the cutting edge. There’s visibility like you’ve never seen before. When you’re sitting in a loader, you can see underneath the bucket as you’re loading or dumping."

In Teleo’s latest user interface, there is a bubble on the screen that turns green when the operator is in full control, and a blue bubble indicates that a machine is travelling from point A to point B by itself.

“When that machine sees an obstacle, potentially a human or potentially another hazard, or has reached its destination, that blue bubble turns yellow—looking for human interaction," Piotto says.

"All the while when that human is involved, we are learning how that human interacted with the machine and what they did to avoid the obstacle, or how they dumped the material."

If there is excessive boom up or curl, or poor operating technique, Piotto said Teleo’s AI can detect that, as well. And the operators that take shortcuts or have bad habits will not fool the A.I. learning.

“There’s some intuitiveness in our AI to learn the proper way to operate, becase we know that when we have thousands of machines out there we’re going to have a couple bad apples,” Piotto said. “But those bad apples aren’t going to disrupt the entire learning process.

“This is here today, and in production on live job sites," Piotto said.

Currently, Teleo has supervised autonomy running in Florida, California, and Canada.