Bin picking Digital twin

Project description

Affix Engineering continuously explores ways to improve the efficiency and flexibility of robotic automation. Developing and fine-tuning robotic systems directly in physical environments is time-consuming, costly, and can cause unnecessary wear or damage. This challenge is especially evident in bin-picking applications, where a robotic manipulator must precisely detect, grasp, and place objects into designated bins, requiring extensive testing and optimization. To address this, the project focuses on creating a realistic and interactive virtual environment that accurately mirrors the physical setup.

The simulation includes detailed models of the workspace, robotic arm, and objects, and enables automated training and testing using Isaac Sim. The main research question guiding this work is: How can a robotic manipulator be controlled using the digital twin paradigm in Isaac Sim to perform a bin picking task? The final objective is to deploy the validated control approach on the real robotic system for industrial automation applications worldwide.

Context

One such challenging application is bin picking, where a robotic manipulator must autonomously detect, grasp, and place objects from unstructured environments into predefined locations. Achieving reliable performance requires extensive tuning of perception, motion planning, and control algorithms. To reduce development time and risk, simulation-based development using digital twins is gaining importance in industrial robotics.

This project is situated within this context and focuses on the use of NVIDIA Isaac Sim to create a realistic digital twin of Affix Engineering’s bin-picking setup. The virtual environment mirrors the physical workspace, robotic arm, and handled objects, enabling controlled experimentation and automated testing. By training and validating control strategies in simulation first, the project aims to minimize physical testing iterations. Ultimately, the validated solution will be transferred to the real robotic system, demonstrating the practical value of digital twins for efficient and scalable robotic system development.

Results

The result of the project that were obtained differ from the results found in our initial plan. In the beginning the group planned on implementing a full digital twin environment which would control the robotic manipulator’s movement, however, this proved to be a difficult task within the given scope.

As a result, the final product of this project is a virtual environment with a digital twin of the custom robotic manipulator in Isaac Sim. This environment utilizes different algorithms that can calculate the pick point of object in the bin, as well as the dynamic collision-free trajectory of the manipulator from the pickup location, to the drop location. This project is meaningful to Affix in terms of good foundation layer for software in which complex robotic applications could be trained and tested. In other words, it is an all-in-one solution to the individual problems that the company had, in one place.

About the project group

This project was done by three Fontys students from the Advanced Smart Industry semester the study program, HBO-ICT. Each student has good knowledge in embedded systems with specialties in smaller fields.