The Robot Report

  • Home
  • News
  • Technologies
    • Batteries / Power Supplies
    • Cameras / Imaging / Vision
    • Controllers
    • End Effectors
    • Microprocessors / SoCs
    • Motion Control
    • Sensors
    • Soft Robotics
    • Software / Simulation
  • Development
    • Artificial Intelligence
    • Human Robot Interaction / Haptics
    • Mobility / Navigation
    • Research
  • Robots
    • AGVs
    • AMRs
    • Consumer
    • Collaborative Robots
    • Drones
    • Humanoids
    • Industrial
    • Self-Driving Vehicles
    • Unmanned Maritime Systems
  • Business
    • Financial
      • Investments
      • Mergers & Acquisitions
      • Earnings
    • Markets
      • Agriculture
      • Healthcare
      • Logistics
      • Manufacturing
      • Mining
      • Security
    • RBR50
      • RBR50 Winners 2025
      • RBR50 Winners 2024
      • RBR50 Winners 2023
      • RBR50 Winners 2022
      • RBR50 Winners 2021
  • Resources
    • Automated Warehouse Research Reports
    • Digital Issues
    • eBooks
    • Publications
      • Automated Warehouse
      • Collaborative Robotics Trends
    • Search Robotics Database
    • Videos
    • Webinars / Digital Events
  • Events
    • RoboBusiness
    • Robotics Summit & Expo
    • DeviceTalks
    • R&D 100
    • Robotics Weeks
  • Podcast
    • Episodes
  • Advertise
  • Subscribe

How to maximize machine vision guided robotics

By Scott Strache | Mitsubishi | July 5, 2020


Machine vision systems are increasingly being used to guide robot actions — a trend that has grown out of recent advances in affordable machine vision technologies and industrial computing power. When coupled with 2D or 3D sensors, robots can be made to perform a wide variety of tasks, from basic inspection to more complex pick-and-place operations.

But to truly reap the benefits of vision-guided robotics, you need to select the right system for your application. Today, you have several types of machine vision systems to choose from, each with its own system requirements and sensor technologies. How do you know which one is right for your application? To determine this you need to consider the needs and goals of your operation, including the size and orientation of workpieces, as well as processing times.

In this article, we will consider the criteria that can help you select the best machine vision system for your robotics application. We will also discuss some of the hidden costs associated with 3D machine vision systems, which can help drive your decision.

An overview of 2D machine vision

Before we get to 3D machine vision, let’s review 2D machine vision. 2D machine vision is typically used for inspection tasks, such as checking the dimensions, features, and orientation of parts as they move along production lines. These systems work by generating flat, two-dimensional maps of reflected intensity, or contrast, making lighting an important factor in these applications. Because too much or too little light can throw off the accuracy of  images, it’s important to consider ambient conditions, artificial light and shadows in order to capture part edges and features clearly.

Although having only X and Y data is sufficient for many applications, like simple object tracking, 2D vision systems have their limits. For one, they render real-life, three-dimensional objects as flat, 2D projections with no depth of field. This lack of a third dimension presents a challenge for tasks that rely on object shape and orientation, such as bin picking.

Like 3D machine vision systems, 2D systems are also sensitive to lighting conditions. Natural light sources, such as windows or skylights, can create affect sensor readings. Sometimes adding an enclosure or shroud to reduce these conditions increases the success of these applications.

Even with some limitations, 2D vision systems are cost-effective and easy to implement in many applications. Examples include quality inspection, part detection, optical character recognition, barcode reading and many more.

An introduction to 3D machine vision

Although machine guided robotics can involve 2D sensors, these applications typically use 3D vision systems, which operate in conjunction with higher-performing six-axis or SCARA robots. There are several 3D sensor technologies to choose from, including laser displacement, structured light, and point cloud, which involves generating a list of three-dimensional coordinates to represent an object’s surface in space. The 3D camera generates the point cloud, and then image processing software analyzes the point cloud file to guide the robot.

Mitsubishi vs3d config

Source: Mitsubishi Electric

Unlike 2D sensors, which generate flat images of objects, 3D sensor technologies can guide robots in complex pick- and-place and inspection applications. They can also handle unstructured part orientations.

In terms of setup, you can integrate 3D vision systems with robotic cells in different ways.

For example, you can attach small, lightweight 3D sensors to the robot hand in what’s known as an end-of-arm configuration or mount the camera above the robot with the lens pointing downward at the robot’s workspace.

The speed of these configurations depends on several factors, including processing times and how long it takes to move to the pick location. In terms of their benefits:

  • The end-of-arm configuration is more flexible, allowing the robot to move the camera to inspect parts with unique orientations, as well as areas that are difficult to access. Keep in mind, this configuration can make your process slower because you have to wait for the robot to move before you can capture an image. You therefore need to factor in the robot’s repeatability.
  • Fixed configurations accommodate a larger field of view, as they are not limited by the reach of the robot. In addition, the camera can take pictures while in motion, reducing cycle times. You also don’t have to worry about the robot’s variance because the camera position is always known. These benefits make this configuration the preferred method when possible.

Applications and Benefits

3D vision systems have many advantages—some of which overcome the shortcomings of 2D machine vision, which typically only provides object information in the X and Y dimensions. While it’s true that some 2D systems can infer simple data in three dimensions, they’re mostly limited to the X-Y plane.

e-vs3d Mitsubishi

Source: Mitsubishi Electric

3D systems generate much richer data in all three directions, making them ideal for complex robotic tasks that need to cope with diverse object shapes and orientations. When properly deployed, 3D vision systems are also highly repeatable and can avoid errors due to object location, orientation and presentation to the sensor.

Because 3D vision systems excel at handling the intricacies of three-dimensional workpieces, they’re ideal for applications that are less organized in nature and involve a random presentation of parts.

One example is bin picking, in which the camera detects and analyzes the randomly piled parts in a bin. Using this information, it then guides the robot to pick up individual workpieces for the next step in the production process. 3D vision systems have the capability of picking parts that have variable surface conditions, such as welded parts or parts that need to be deburred.

Another benefit of 3D vision systems is their ability to match parts using registered 3D CAD models. Some 3D systems also offer technologies that allow part matching without the need to compare parts to a CAD model on the fly. Because less processing time is spent than when comparing the sensor image with a CAD model, this “model-less” matching technology strikes a good balance between the ability to pick randomly oriented parts and also pick speed (see below).

Model-less versus model-matching modes

The MELFA 3D machine vision system allows users to choose between model recognition and model-less modes for robotic workpiece gripping:

  • Model-less: A recognition method that registers the shape of the hand or suction pads and then matches the hand shape to suitable grip locations on the part. Registering the workpiece shape as 3D CAD models is not required.
  • Modelmatching: A recognition method that registers workpiece shapes as 3D CAD models. It then searches for workpieces that match these models in order to identify workpiece posture and grip location.

Hidden cost considerations

Despite the many benefits, there are some cost considerations associated with 3D machine vision systems, and not all of these costs are related to the vision hardware itself. For one, 3D vision often involves extra programming and integration requirements, as well as requiring higher-quality CAD data. You also need to account for the cost and complexity of any auxiliary components like end-of-arm tooling. These tools, which can drive up cost in any robotic system, include suction pad or parallel grippers, sensors and welding torches, as well as tables, fixturing and 2D sensors.

This scenario could apply to any machine vision system, but 3D systems can exacerbate this issue due to the processing time added by CAD model matching.

That being said, if you want to minimize the hidden costs of 3D vision and maximize its benefits, it’s important to pick the right type of vision system for the job at hand. To illustrate how different systems target different applications, consider two 3D systems:

  • The MELFA 3D system is a clear choice for applications that don’t require model matching to grab and orient parts. It is also ideal for smaller part sizes and features good part orientation capabilities.
  • The Canon system excels whenever full model matching is needed to meet the part handling objectives. It can also handle larger parts and bins than the MELFA system due to its partial CAD recognition feature, which allows the system to recognize a part that is not entirely within the camera’s field of view.

The right machine vision system for you

Picking the right machine vision system for your application is a complex topic that typically requires some engineering hours. However, the fundamentals boil down to a combination of part size, diversity and orientation, requirements for robot processing times and system costs. Oftentimes, these selection criteria will point you in a clear direction when it comes time to selecting a 3D vision system.

To learn more about how to select the best machine vision system for your application, visit:
https://us.mitsubishielectric.com/fa/en/products/industrial-robots-melfa/intelligent-options.

Scott Strache, Mitsubishi ElectricAbout the author

Scott Strache is a product manager for robotics at Mitsubishi Electric Automation Inc.

Tell Us What You Think! Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Related Articles Read More >

New enabling technologies from Automate 2025
The Gemini 435Le sensor package from Orbbec.
Orbbec designs Gemini 435Le to help robots see farther, navigate smarter
Images showing the difference between the CARLA simulator on its own, and working with Helm.ai's GenSim-2.
Helm.ai launches AV software for up SAE L4 autonomous driving
A talk being given at the 2024 Robotics Summit's Engineering Theater.
9 sessions to see at the Robotics Summit Engineering Theater

RBR50 Innovation Awards

“rr
EXPAND YOUR KNOWLEDGE AND STAY CONNECTED
Get the latest info on technologies, tools and strategies for Robotics Professionals.
The Robot Report Listing Database

Latest Episode of The Robot Report Podcast

Automated Warehouse Research Reports

Sponsored Content

  • Sager Electronics and its partners, logos shown here, will exhibit at the 2025 Robotics Summit & Expo. Sager Electronics to exhibit at the Robotics Summit & Expo
  • The Shift in Robotics: How Visual Perception is Separating Winners from the Pack
  • An AutoStore automated storage and retrieval grid. Webinar to provide automated storage and retrieval adoption advice
  • Smaller, tougher devices for evolving demands
  • Modular motors and gearboxes make product development simple
The Robot Report
  • Mobile Robot Guide
  • Collaborative Robotics Trends
  • Field Robotics Forum
  • Healthcare Robotics Engineering Forum
  • RoboBusiness Event
  • Robotics Summit & Expo
  • About The Robot Report
  • Subscribe
  • Contact Us

Copyright © 2025 WTWH Media LLC. All Rights Reserved. The material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with the prior written permission of WTWH Media
Privacy Policy | Advertising | About Us

Search The Robot Report

  • Home
  • News
  • Technologies
    • Batteries / Power Supplies
    • Cameras / Imaging / Vision
    • Controllers
    • End Effectors
    • Microprocessors / SoCs
    • Motion Control
    • Sensors
    • Soft Robotics
    • Software / Simulation
  • Development
    • Artificial Intelligence
    • Human Robot Interaction / Haptics
    • Mobility / Navigation
    • Research
  • Robots
    • AGVs
    • AMRs
    • Consumer
    • Collaborative Robots
    • Drones
    • Humanoids
    • Industrial
    • Self-Driving Vehicles
    • Unmanned Maritime Systems
  • Business
    • Financial
      • Investments
      • Mergers & Acquisitions
      • Earnings
    • Markets
      • Agriculture
      • Healthcare
      • Logistics
      • Manufacturing
      • Mining
      • Security
    • RBR50
      • RBR50 Winners 2025
      • RBR50 Winners 2024
      • RBR50 Winners 2023
      • RBR50 Winners 2022
      • RBR50 Winners 2021
  • Resources
    • Automated Warehouse Research Reports
    • Digital Issues
    • eBooks
    • Publications
      • Automated Warehouse
      • Collaborative Robotics Trends
    • Search Robotics Database
    • Videos
    • Webinars / Digital Events
  • Events
    • RoboBusiness
    • Robotics Summit & Expo
    • DeviceTalks
    • R&D 100
    • Robotics Weeks
  • Podcast
    • Episodes
  • Advertise
  • Subscribe