A simulated production line let its AI choose the next plant state rather than each motor command.
A fixed inverse model translated that target into settings for five actuators.
In one test, training fell from 100,000 cycles to 30,000.
Researchers in Germany have tested a different way to train artificial intelligence for modular factories. Instead of asking a reinforcement-learning agent to set every conveyor and pump directly, they trained it to specify what the production line should look like one step later. A separate model then converted that desired state into machine commands.
The experiment used a simulation of a four-module bulk-material plant. Material is loaded, stored, weighed and filled into containers. Five controllers adjust equipment including conveyors and a vacuum pump while watching the fill levels in neighbouring reservoirs. A poor decision can stop the flow or cause material to overflow.
The researchers first learned a forward model from operating data. It predicts the next fill levels from the current state and the actuator commands. They also trained an inverse model: it receives the current fill levels and the target for the next moment, then calculates the motor speeds and pump timing required to reach that target. Each inverse network contained about 17,000 parameters and was frozen before reinforcement learning began.
This split makes the learning problem easier to search. The AI chooses a physically meaningful destination, while the inverse model handles the mixed continuous and on-off controls beneath it. Across three reinforcement-learning algorithms and two production rates, the inverse-model version reduced both overflow and the number of training cycles. At a flow rate of 0.15 litres per second, one DDPG run reached its target in 30,000 cycles instead of 100,000. In a TD3 test at the same rate, overflow fell from 0.137 litres to 0.009 litres.
The numbers come from a laboratory-scale simulator, not a working factory. The study used one-step prediction and did not report repeated trials with multiple random seeds and confidence intervals. Real equipment introduces sensor drift, wear, network delays and safety limits that a simulator may not reproduce.
The next useful test is therefore operational: rearrange the modules or replace an actuator, then measure how many trial runs and how much downtime are needed before production stabilises again. If the method continues to reduce overflow while maintaining throughput and energy use, it could shorten one of flexible manufacturing's costly tasks: retuning a line whenever the product or layout changes.
Sources
Distributed Optimization of Modular Production Systems Using Model-Based Reinforcement Learning with Inverse Models
MLPro documentation for the Bulk Good Laboratory Plant