University of Limerick Pilots Intelligent ROVs

Researchers at the University of Limerick are developing control algorithms to enable remotely operated vehicles to be more autonomous.

The Mobile & Marine Robotics Research Centre at the Irish university named its remotely operated vehicle (ROV) “Latis” after a Celtic god of water and beer, said Gerard Dooly, a research fellow and postdoc student at the MMRRC.

Navigation gets an automated assist

“The idea is to take the sole dependence on the pilot out of the control loop, leading to benefits such as the ROV more quickly reacting to conditions, cut expensive ship time, and operate in more types of weather,” said Dooly. “The pilot in the control loop would simply send top-level commands.”

“It has full INS, and it uses dead reckoning to estimate it?s position underwater,” Dooly told Robotics Business Review during a recent visit to the school’s campus.

“Our field research is more focused on working with humans than on AI or full automation,” said Daniel Toal, director of the center.

In addition, the software can “dislocate heading from direction,” said Dooly, allowing it to autonomously maintain heading while moving in another direction. For instance, Latis can keep sonar or a high-definition camera trained on an object even as it circles the object or keeps a constant distance from a moving ship.

A team successfully used Latis to survey submerged tanks as part of a test with the Irish Coast Guard and the Marine Institute.

Eden Omerdic shows remote operation of OceanRINGS.

“The INS position data is updated 10 times per second, and the ROV corrects thruster commands autonomously, based on top-level commands, once per second, allowing it to react quickly and keep an accurate course or position, simplifying the pilot experience,” Dooly explained. “Instead of requiring months of training, a pilot can be trained in a few hours.”

The ROV can automatically respond to changing currents and angled trajectories, said Eden Omerdic, a senior research fellow at the MMRRC.

During a 2013 test in the Shannon Estuary, Latis was able to stay on course even when selected thrusters were shut off. Fully manual operation wouldn’t be able to make the needed corrections without overcompensating, he noted.

Latis was able to survey a vessel in a way that could help determine whether similar stranded vessels can be recovered or are too fragile.

The ROV’s software is also Internet-enabled, so a pilot could sit at a command center in, say, Abu Dhabi and control an ROV near Ireland, Omerdic said. Thanks to the local autonomous controls, there are fewer demands on bandwidth for piloting.

WetLab tests ROV hardware

“We built Latis from scratch,” said Dooly. The MMRRC spent approximately $1.1 million on the full system build, which included personnel costs. The ROV is equipped with a state-of-the-art multibeam echo sounder from Teledyne Reson, capable of producing a precise map of the seafloor as the ROV moves forward. The system is not affected by turbidity as other imaging sensors are and is used throughout industry where resolution is a neccessity.

The MMRRC has a “DryLab,” where the software and manipulators are developed, and a “WetLab” for ROV testing. “We could use a bigger tank,” Dooly acknowledged.

Latis‘ buoyancy system is rated to 1 km (0.62 mile) in depth, although some of its instruments are rated to 6 km.

The design and development process took about 18 months, and the system has been offshore 11 times in the past six years.

A control cabin in a 6-ft. truck container has a power supply and workstations for the pilot and sonar operator. The control algorithms take GPS coordinates and feed them via fiber optics to the ROV so that it can triangulate its position relative to the ship.

In addition, the MMRRC plans to upgrade Latis’ hardware. The researchers have access to Schilling and Staubli robotic arms for testing and teaching, respectively, and they plan to work toward smaller, lighter arms for mounting on their ROV.

Machine vision and 3D are easier in industrial settings than underwater, noted Dooly, who is also a qualified closed-circuit trimix rebreather diver. He has personally descended to depths of up to 135 meters (442 feet), and among other noteworthy shipwrecks, he has explored the Lusitania, one of many ships sunk around Ireland during the World Wars.

This practical experience has highlighted the need for ROVs that are aware of their surroundings and that can let pilots focus on objectives. “We are looking at adaptive controls, neural nets, and deep learning to improve control,” Dooly said.

Steadily shrinking high-def cameras should also help, said Toal, “especially in the ocean, where more is hidden.”

Simulation software, varied controls to save money

The first time the MMRRC took Latis out to sea for testing, it learned that such cruises are expensive, said Dooly. Renting a vessel to bring the ROV and control container can cost more than €5,000 ($5,520) per day, plus crew costs. Rather than spend days in port testing integration aboard a ship, the MMRRC has developed simulation software.

“We want to test virtually with real signal flow before going on ship,” said Omerdic. “We developed new algorithms to simulate different motion.”

As part of its Internet-enabled controls, the University of Limerick’s team has created a “virtual control cabinet” with a relatively simple user interface.

“The OceanRINGS software is a generic solution,” said Omerdic. “We wanted to account for all possible thruster configurations and allow for tele-operation.”

He compared a recorded simulation with actual videos demonstrating that Latis could follow a circular course or a zig-zag search pattern with a minimum of oscillation from course corrections due to water currents.

Not only does OceanRINGS enable semi-automated ROV control; it’s also intended to allow for the management of multiple drones and the data that they gather.

“We also provide for varied input devices, including tablet touchscreens, video game controllers, and voice commands,” Omerdic said. “Most of the controls are local and automated, so with a minimum reliable bandwidth of 0.7 Mbit/sec., pilots can observe the situation and react in real time.”

In addition, feedback from joysticks can inform pilots when they reach “saturation boundaries,” i.e., when a chosen vector has passed maximum thruster effectiveness. Omerdic said he is working on improving algorithms so that “the auto-tuning, low-level controllers have a wider saturation boundary and waste less energy.”

“We had a software demonstration in cooperation with Memorial University in St. John’s, Newfoundland,” said Omerdic. “It was a competition between an ROV operations instructor and a student who had been quickly trained to use our semi-manual system. With automation, the student did better than the instructor.”

Memorial University is conducting demonstrations to see if the University of Limerick’s software will work on its own machinery. Other institutions can get access to the MMRRC’s simulator to develop and test their own algorithms, which could be adapted to other hardware, Omerdic said.

“We have the best mathematical model for thruster fault detection and accommodation,” Omerdic said.

Gerard Dooly and Latis at the University of Limerick.

Using the MMRRC’s systems, a pilot could even control an ROV through the Internet with voice commands alone, said Omerdic. He mentioned being at an Oceanology International conference in Abu Dhabi in the United Arab Emirates and showing students how to control a drone long-distance.

ROVs are often rented as services by energy companies, but Dooly and Omerdic foresee that easier piloting and smaller drones could allow for easier inspections. For instance, an oil rig or wind turbine could have a container on its support struts that would deploy a small, tethered ROV without waiting for a ship, and then reel it in when done.

“We’re working on building a ‘garage’ for some of the smaller ROVs,” said Dooly.

Research has practical goals

The MMRRC’s refinements to Latis’ operations have a practical goal — work-class ROVs to inspect and repair offshore oil rigs and tidal turbines. Such ROVs could enable long-term, low-cost monitoring of offshore installations.

Ireland is No. 1 in Europe for marine renewable energy, which is not just an economic issue, but also a security one, said Dooly.

“A significant savings in the time required to perform underwater tasks, along with the commensurate savings in operational costs, would be achieved should this technology become available for wide-scale commercial and industrial use,” said Richard VanderVoort, chief of ROV operations at Memorial University, in an OceanRINGS statement.

The MMRRC also hopes to study autonomous, untethered drones and fleets of drones, similar to those being developed for mine detection and search and rescue in Italy for NATO.

In addition, the center is investigating the use of tethered parafoil kites for airborne wind energy and sensor/communications platforms, said Toal. The European Union is supporting research into marine renewable energy, while Google is working on fixed-wing airborne power generation.

Dooly said he expects Latis’ technologies to be useful for anti-mine measures, detecting columns of oil spills in bodies of water, and helping maintain renewable energy stations.

The MMRRC has a practical approach and is looking for research and industrial partners, said Toal. Its current partners include the Irish navy, which is committed to advancing the use of robotics without creating automated weapons platforms.

The MMRRC has four Ph.D. candidates, four postdoctoral students, and summer interns from Italy and France under the EU’s Erasmus exchange program. The University of Limerick says that it has the largest work placement of any university in Ireland, with a network of more than 1,700 employers and graduate employment 15 percent above the national average.

The MMRRC is “planning to transform underwater ROV operations in much the same way that autopilot transformed air travel,” said EuroAsia Industry.