One of the biggest challenges in robotics is developing autonomous systems that can reliably operate in unmapped environments. This is why, for example, leading self-driving car company Waymo only tests in areas it’s spent countless hours at creating detailed 3D maps.

Exyn Technologies, a spin-off of the University of Pennsylvania’s GRASP lab founded by Vijay Kumar, is making tremendous in-roads in autonomous navigation and real-time mapping of unfamiliar, infrastructure-less environments that don’t have beacons, GPS or ultrawideband technology. The technology works just as well in environments equipped with infrastructure, but that’s old-hat at this point.

The Philadelphia startup, which took part in our inaugural Robotics Summit & Showcase, is developing scalable, full-stack software called exynAI that enables autonomous navigation and real-time mapping for Autonomous Aerial Robots (A3R) in complex commercial environments. At the moment, Exyn is focusing on indoor drone applications, including construction, logistics and mining. But the technology can also work on ground-based robots. In fact, Exyn is currently working with a legged robotics company, but it can’t publicly disclose the name of its partner just yet.

How exynAI works

So how does it work? Exyn co-founder Nader Elm said the key to enabling autonomous navigation starts with these questions:

• Where am I in the environment?
• Where am I trying to go?
• How do I get there optimally?

Exyn uses multiple sensors to perceive its environment – cameras, LiDAR (Velodyne VLP-16), radar, and RGBD – and late-stage sensor fusion that compiles the data in real time to estimate a robot’s position and orientation relative to where it started. The VLP-16 is the primary sensor for SLAM and for part of the system’s state estimation capabilities. “Each sensor gives state estimation, then we fuse it,” Elm said. “Depending on the confidence level of each sensor, we’ll use it. If it’s low confidence, we consider it bad data and get rid of it.”

Check out the video below of Exyn demonstrating its autonomous flight capabilities in a mine. According to Exyn, the route was not pre-programmed, human pilots did not manipulate controls, and processing is done onboard the drone in real time.

The difficult part, Elm said, is building the AI that helps the A3R understand the environment they’re seeing in 3D. Exyn uses machine learning and deep neural networks to do so. “This is the hard part Google has been working on for a decade. What Google is doing to make cars driverless, we’re doing for drones. Exyn has an advantage of leveraging a decade of research from UPenn.”

“As the drone is moving and obstacles dynamically appear in its path — a forklift or a person — you need to dynamically change the path. We can do that in real time with onboard processing,” Elm said. “But now, using machine learning, we can identify what’s in front of the drone and change the behavior of the drone based on what it is. If it’s a forklift, for example, go around it and complete mission. If it’s a person, come to a stop and stay a safe distance away. These are new and exciting things we’re integrating into our stack – semantic labeling of objects in the drone’s path.”

Exyn said the exynAI software can be customized to be mission-specific and supports swarms. It also provides a “sliding scale” of autonomy, ranging from fully autonomous flights to self-piloted and pilot-assisted flights with obstacle avoidance.

Why Exyn started with indoor drones

Exyn’s technology appears to be a fit for a number of outdoor drone applications, too. But on-going battles over outdoor, commercial drone regulations have been a turnoff for the company.

“Regulations really constrain outdoor drone applications,” said Elm. “Regulations don’t really exist for indoor drones. Fortune 500 companies will have policies, and we are mindful of people and safety policies in warehouses. Indoor drones are unregulated, but not unrestricted.”

One competitor of Exyn is Australian company Emesent. The company develops Hovermap, which uses LiDAR-based 3D SLAM in real time to allow navigation without GPS. Emesent primarily targets mines and other underground environments.

Two of competitors could be Near Earth Autonomy (NEA) and Skydio, both focus heavily on outdoor applications. Skydio’s R1 is a consumer drone billed as a “flying camera,” while NEA has built systems to inspect agricultural environments, power plants, and many other outdoor areas. Other potential competitors include Airobotics and Iris Automation.

Indoor Applications

Elm said Exyn technology is currently in the pilot stage, “looking to put more miles on the system and make it even more reliable.” Exyn isn’t a drone company, it’s a software company building AI that enables drones to complete tasks without human intervention. The initial three markets for A3R are construction, logistics and mining. Here’s a quick look at each of these applications:

Construction: an inside-out view
Billions of dollars are lost each year in the construction industry due to difference between how a building, for example, is designed and how the actual construction pans out. This leads to projects being behind schedule and over budget. There are companies that build 3D models of buildings from the outside, including 3D Robotics for example, but Elm said “the majority of construction problems occur inside.” So Exyn provides an inside-out view on construction problems.

“As a roof deck goes on and GPS becomes unavailable, we can provide maps and models from the inside of the building on a regular basis to compare as-is against the design plans,” he said. “This allows the construction team to track progress and find errors and address reworks. Imagine big metal beams holding up the roof. They all look the same. But when the roofing guys arrive, they notice one beam is too tall, one beam is too small. Now the roof is crooked. This is a $50,000 fix and is a week delay.”

Mine surveying
Mine surveying is a critical task necessary for designing, excavating, and documenting information at all stages of mining. But mines are dangerous environments for humans, and surveying techniques are labor intensive and time consuming.

Exyn’s A3R robots, again with no pre-loaded maps or GPS, can be deployed anywhere within a mine and be used to map the environment and collect data.

“Imagine a robot building a map of a mine on a frequent basis to track layout of mine and 3D models to understand volumes of what’s been extracted?? These are specific pain points in mining that we’re looking to automate for data acquisition.”

Warehouse inventory management

Drones are starting to make their way into warehouses for inventory management applications, and the A3R can communicate in real time with warehouse management systems. For larger sites, collaborative robot swarms can be used to perform inventory management quicker.

“Vision-only systems require the lights to stay on, which isn’t ideal for the warehouse owner,” Elm said. “The systems also need to operate in very narrow aisles, so the system needs very good state estimation. So typically this would require infrastructure, but we don’t require any of that.

“You’ll be going down aisles and the lights go out. Warehouses are motion activated, and the drones don’t activate the lights. So you need to be able to operate in both types of environments and transition between both environments smoothly.”