3D-Sensors

With AI and Open Source to new possibilities in object recognition

The thematic context

On the way to fully flexible and automated logistics, topics such as sensor technology and object recognition are of particular relevance. For example, the reliable recognition of objects in sensor data is a basic requirement for the automation of logistics processes. In particular, the recognition of objects as 3D sensor data is essential for flexible autonomous applications. However, in addition to high development costs for new solutions, previous solutions in 3D object recognition are not very flexible because they are difficult to transfer to different applications. Innovative solutions are needed to meet these challenges.

The demonstrator

The 3D sensor technology demonstrator shows a 3D object recognition and the picking of different, not sorted articles from the retail and e-commerce sector out of a container. The use case is based on a typical problem from the field of bin picking or belt picking (separation of objects on a conveyor belt).

Visualisierung-Demonstrator — © Fraunhofer IML - Sebastian Hoose

The technical solution

The randomly arranged objects are detected by a 3D sensor. The sensor data is then evaluated with the aid of artificial intelligence (AI). For this purpose, the AI performs a pixel-by-pixel classification and segmentation (instance segmentation) of the various objects in the container. For a randomly selected object, possible gripping positions are calculated from the segmented 3D data. A lightweight robot with suction gripper then picks up the specific object from the bin and places it on the adjacent table. In addition to standard object localization, object classification takes place thanks to AI. The special feature: Both the software for 3D object recognition and the gripper function were developed on the basis of open source components.

Our skills

Our qualifications lie in the concept development of new and simple solutions for the logistic and industrial context in the specialist area of 3D object recognition and industrial robotics. The focus of our work is the selection, adaptation and integration of open source software for 3D object recognition as well as for gripping with lightweight robot arms for logistic use cases. But also the development of completely new solutions, based on open source, are in our area of qualification. Regarding the presented 3D sensor technology demonstrator, we have the competences for the technical support of the demonstrator to a product. Furthermore, we are also available for the further development of the demonstrator to another use case.

Your benefits

By consistently using and further developing open source software and components, the development costs for new solutions are significantly lower. Based on standardized interfaces (in ROS) to the 3D sensor, other sensors or robots can be flexibly integrated. Thanks to the newly gained flexibility and the innovative AI approach, simple extensions to new use cases and applications (palletizing, depalletizing, semantic mapping) can also be realized.

You have become interested?

...then get in touch with our contact persons listed opposite!