The Robot Report

  • Home
  • News
  • Technologies
    • Batteries / Power Supplies
    • Cameras / Imaging / Vision
    • Controllers
    • End Effectors
    • Microprocessors / SoCs
    • Motion Control
    • Sensors
    • Soft Robotics
    • Software / Simulation
  • Development
    • Artificial Intelligence
    • Human Robot Interaction / Haptics
    • Mobility / Navigation
    • Research
  • Robots
    • AGVs
    • AMRs
    • Consumer
    • Collaborative Robots
    • Drones
    • Humanoids
    • Industrial
    • Self-Driving Vehicles
    • Unmanned Maritime Systems
  • Business
    • Financial
      • Investments
      • Mergers & Acquisitions
      • Earnings
    • Markets
      • Agriculture
      • Healthcare
      • Logistics
      • Manufacturing
      • Mining
      • Security
    • RBR50
      • RBR50 Winners 2025
      • RBR50 Winners 2024
      • RBR50 Winners 2023
      • RBR50 Winners 2022
      • RBR50 Winners 2021
  • Resources
    • Automated Warehouse Research Reports
    • Digital Issues
    • eBooks
    • Publications
      • Automated Warehouse
      • Collaborative Robotics Trends
    • Search Robotics Database
    • Videos
    • Webinars / Digital Events
  • Events
    • RoboBusiness
    • Robotics Summit & Expo
    • DeviceTalks
    • R&D 100
    • Robotics Weeks
  • Podcast
    • Episodes
  • Advertise
  • Subscribe

Obstacle avoidance to get sharper vision with UMass Amherst research

By Sabrina Guillen | September 2, 2019

Obstacle avoidance with Jackal UGV

The Introspective Vision for Obstacle Avoidance at UMass Amherst uses the Jackal UGV to train machine learning. Source: Clearpath Robotics

Humans are dependent on their senses. Without external inputs from the outside world, we are unable to locate, identify, navigate, operate, and much more. Similarly, robots require advanced perception for obstacle avoidance and to function in a world designed by and for humans. However, while vision is an inexpensive sensing system for robots, it is prone to errors for various causes such as reflective surfaces, blur, and texture-less scenes.

A research duo at the College of Information and Computer Sciences at the University of Massachusetts Amherst is using the Jackal unmanned ground vehicle (UGV) from Clearpath Robotics Inc. to train competency-aware, vision-based obstacle avoidance systems that can predict such failures. This would allow for less-expensive and safer robot deployments. It is only through learning competence-aware perception algorithms that one can predict their failure cases and reason about their types of failure.

In their Introspective Vision for Obstacle Avoidance (IVOA) project, Ph.D. student Sadegh Rabiee and Assistant Professor Joydeep Biswas are focusing on learning the competency of computer vision algorithms for the task of obstacle avoidance.

Giving Jackal eyes for obstacle avoidance

To prepare the Jackal UGV for the task, Rabiee and Biswas equipped the platform with a pair of stereo cameras, as well as a depth camera to provide sparse ground truth. These would be used to log RGB and depth images at full frame rate during deployments of the robot. Those images are then separately processed for obstacle avoidance.

When the plans generated by the two disagree, and when the environment is known to be one where the depth sensor is more reliable, the point where the paths diverge is projected onto the image plane of the RGB image. Consequently, an image patch centered at that location is extracted as an example of unreliable image conditions.

By using such image patches that yield failures of the image-based algorithm, the team can train an introspection model to learn to predict the following:

  • Will an input image yield failures?
  • Which parts of the image are likely to cause such failures?
  • What type of failure is expected? (false positive vs. false negative)
  • How many distinct classes of failures were extracted from the training set?

Tailoring Jackal UGV for research in long-term autonomy for mobile robots required the UMass team to customize the platform with their own desired sensors and payloads. To begin, they attached a stereo pair of Point Grey cameras for vision-based SLAM (simultaneous localization and mapping) and obstacle avoidance, as well as Kinect depth sensors to provide ground truth for their vision-based obstacle avoidance system.

Next, a Velodyne Lidar (VLP 16) was used for research on lidar-based SLAM, as well as ground truth generation for vision-based SLAM.

The final pieces of the hardware puzzle included a touch screen monitor for human-robot interaction and an Intel NUC for additional computing power. The touch screen enables quick and subtle debugging, as well as allowing the robot to interact with humans via a graphical user interface. An example of this would be a robot seeking help from people nearby for using the elevator.

In terms of software, the IVOA team developed their own complete ROS-based stack for autonomous navigation of the robot, including SLAM implementation, obstacle avoidance, and planning.

Fitting Clearpath into the equation

Where does Clearpath Robotics fit in? Well, IVOA’s data collection process requires having a mobile robot platform capable of operating reliably for extended periods of time in different environments and traversing different types of terrain.

At the same time, IVOA relies on extensive self-labeled training data, which would have been infeasible to collect manually. Using Jackal UGV enabled the acceleration of their research process in three ways:

  1. Eliminating the need to build and maintain their own UGV. This would have required a significant amount of time and effort
  2. Providing a flexible, modular platform with hardware and software support to easily experiment with different sensor configurations
  3. Proving to be a robust, low-maintenance research platform

The IVOA team were already confident in using Jackal UGV as they had done so in the past for their project titled “A Friction-Based Kinematic Model for Skid-Steer Wheeled Mobile Robots”, which was published at ICRA 2019 (IEEE International Conference on Robotics and Automation).

The researchers chose Jackal UGV because it was a good fit for developing their algorithms and eliminated the need to design and build their own robot, ultimately allowing for a smooth experience when conducting experiments for their research.

“Clearpath Robotics makes research in robotics easier via providing reliable robot platforms that are easy to customize,” said Rabiee. “Also, we have found the support team at Clearpath to be very responsive and helpful”.

Looking ahead

The IVOA project continues to be improved upon, and the team has plans to extend its work on competence-aware perception systems to more perception problems such as vision-based SLAM. The researchers’ ultimate goal is to deploy their autonomous robots at the university campus scale and have them perform safe navigation via competence-aware vision systems.

Currently, they have been able to successfully introduce IVOA, an architecture for self-aware, stereo vision-based obstacle avoidance systems capable of predicting their failures, while distinguishing between false positive and false negative instances. This project will also appear at IROS 2019 (IEEE/RSJ International Conference on Intelligent Robots and Systems).


The Robot Report has launched the Healthcare Robotics Engineering Forum, which will be on Dec. 9-10 in Santa Clara, Calif. The conference and expo focuses on improving the design, development, and manufacture of next-generation healthcare robots. Learn more about the Healthcare Robotics Engineering Forum.


Tell Us What You Think! Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Related Articles Read More >

The Star Wars licensed G1T4-M1N1 droid.
Piaggio Fast Forward launches Star Wars licensed droid
The Gemini 435Le sensor package from Orbbec.
Orbbec designs Gemini 435Le to help robots see farther, navigate smarter
An image of ABB's Flexly P604 Visual Slam AMR.
ABB upgrades Flexley Mover AMR with visual SLAM capabilities
An Aurora ruck driving on a road in Texas.
Aurora begins driverless commercial trucking in Texas

RBR50 Innovation Awards

“rr
EXPAND YOUR KNOWLEDGE AND STAY CONNECTED
Get the latest info on technologies, tools and strategies for Robotics Professionals.
The Robot Report Listing Database

Latest Episode of The Robot Report Podcast

Automated Warehouse Research Reports

Sponsored Content

  • Sager Electronics and its partners, logos shown here, will exhibit at the 2025 Robotics Summit & Expo. Sager Electronics to exhibit at the Robotics Summit & Expo
  • The Shift in Robotics: How Visual Perception is Separating Winners from the Pack
  • An AutoStore automated storage and retrieval grid. Webinar to provide automated storage and retrieval adoption advice
  • Smaller, tougher devices for evolving demands
  • Modular motors and gearboxes make product development simple
The Robot Report
  • Mobile Robot Guide
  • Collaborative Robotics Trends
  • Field Robotics Forum
  • Healthcare Robotics Engineering Forum
  • RoboBusiness Event
  • Robotics Summit & Expo
  • About The Robot Report
  • Subscribe
  • Contact Us

Copyright © 2025 WTWH Media LLC. All Rights Reserved. The material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with the prior written permission of WTWH Media
Privacy Policy | Advertising | About Us

Search The Robot Report

  • Home
  • News
  • Technologies
    • Batteries / Power Supplies
    • Cameras / Imaging / Vision
    • Controllers
    • End Effectors
    • Microprocessors / SoCs
    • Motion Control
    • Sensors
    • Soft Robotics
    • Software / Simulation
  • Development
    • Artificial Intelligence
    • Human Robot Interaction / Haptics
    • Mobility / Navigation
    • Research
  • Robots
    • AGVs
    • AMRs
    • Consumer
    • Collaborative Robots
    • Drones
    • Humanoids
    • Industrial
    • Self-Driving Vehicles
    • Unmanned Maritime Systems
  • Business
    • Financial
      • Investments
      • Mergers & Acquisitions
      • Earnings
    • Markets
      • Agriculture
      • Healthcare
      • Logistics
      • Manufacturing
      • Mining
      • Security
    • RBR50
      • RBR50 Winners 2025
      • RBR50 Winners 2024
      • RBR50 Winners 2023
      • RBR50 Winners 2022
      • RBR50 Winners 2021
  • Resources
    • Automated Warehouse Research Reports
    • Digital Issues
    • eBooks
    • Publications
      • Automated Warehouse
      • Collaborative Robotics Trends
    • Search Robotics Database
    • Videos
    • Webinars / Digital Events
  • Events
    • RoboBusiness
    • Robotics Summit & Expo
    • DeviceTalks
    • R&D 100
    • Robotics Weeks
  • Podcast
    • Episodes
  • Advertise
  • Subscribe