Artificial Intelligence Becomes a Student for Reality

The simulation in question is a sort of school for autonomous robots. In a protected environment, researchers can present robots with problems that they may encounter in the real world. The most challenging thing about the real world is that it is not always possible to make predictions and statistical calculations of what’s going to happen. This is why the researchers make the AI react to as many situations as possible, by simulating variable physical properties, such as images or friction values for the floor, that can be picked up by the sensors. They also use varied structure and load scenarios in the simulation models to ensure the robots are as well prepared as possible for the various logistics processes they will encounter in reality and to make sure the robots can be used in the physical system without issue. In order to keep the modeling complexity at a low level, a process is being developed that can automatically and randomly generate these models. If the AI does its job well, its behavior will be given a positive evaluation and it will react similarly to this situation when it next encounters it. This way of teaching an AI is also known as deep reinforcement learning. Scientists at the Lamarr Institute for Machine Learning and Artificial Intelligence in Dortmund have proven that this method works. The Lamarr Institute focuses on researching and developing powerful, trustworthy, resource-efficient AI applications. As one of the Lamarr Institute’s four scientific partners, Fraunhofer IML is helping to research the optimal design for embodied AI and how it can be used in logistics.

The evoBOT, an agile, high-speed robot, is a model student in the school of embodied AI. The evoBOT belongs to a new generation of autonomous robot systems and is able to assist warehouse employees in all kinds of situations – by handing things to them, for example. Its inverted pendulum design allows it to be used for a wide range of applications; the down side, however, is that it must constantly balance itself when in motion. One aspect of the evoBot that researchers are focusing on at the Lamarr Institute is the way it learns to take its first steps through deep reinforcement learning in a simulation.

In addition to investigating individual embodied robots, the Lamarr Institute team is also researching robot swarms – because in the warehouses of the future, there won’t just be a single robot moving through the halls. While many existing systems control swarms of robots from a central control point, the approach used in Dortmund is different. The control system for the robots is decentralized, which has some huge advantages, especially in terms of flexibility and vulnerability to breakdowns. In the future, the researchers also want to make it possible for the robots to communicate with each other, because these systems also learn through deep reinforcement learning. Each individual robot must continuously analyze its environment in order to operate efficiently and avoid collisions. However, realistically modeling the sensors that carry out this constant analysis in the simulation is an incredibly expensive, time-consuming process. Instead of using a 3D model, researchers at Fraunhofer IML have therefore created a simpler 2D model so that the robot can more quickly learn how to avoid collisions. This has worked well for some scenarios, but it will need further refining in the future, because, as these two projects have clearly shown, the researchers are also facing another, very different problem.

After countless hours in the AI school, the simulated reality, the time has finally come – soon, the embodied AI will “graduate” and take its first steps in real life. However, it cannot simply be let loose into the world. Before that, the robot must overcome the sim-to-real gap, i.e., the discrepancy between simulations and reality. Only then can the robots act and interact independently in the physical world. The preparation for bridging this gap begins back in the simulation, because having an optimal simulated learning environment is key to the robot’s success in the real world. To create this environment, researchers must use complex physical models and a huge number of variants. In order for the behaviors that the robot learns through the simulation to become as ingrained as possible, thereby enabling it to apply them in the real world, the AI must be presented with unexpected situations and put through a lot of training. Creating a simulated world that is as close as possible to the real world is one of the last challenges we face on the path to the digital continuum. The researchers at Fraunhofer IML have already tackled this challenge with their mobile robot, O3dyn. As this robot transports pallets at high speeds, it needed to be trained in the most accurate simulation possible, so it could learn to handle any potential hazardous situations. The results have been amazing: The researchers can send the same movement commands to the real robot and the simulated robot and they will move in exactly the same way. One particular challenge in developing O3dyn was its air suspension, which was initially very stiff and quite unrealistic. It also took some time to create a realistic representation of the omni-directional chassis with its mecanum wheels. Today, by using modern simulation tools and movement data from the actual robot, the researchers have created an advanced simulation that can also depict the robot’s speed. Interfaces and sensors, such as camera and LiDAR data, are also simulated. This means the simulation model is perfect for developing hardware and software in parallel or for testing different application scenarios.