Intelligent PID Tuning with Deep Reinforcement Learning

iStock image: Olemedia

Background

Proportional-Integral-Derivative (PID) is a control algorithm to regulate and stabilise processes. PID control accounts for over 95% of all industrial automation and covers a wide range of domains including optical networking, robotics and automotives. However, its effectiveness is limited by how well tuned the parameters are.

Conventional tuning processes require exhaustive experimentation with numerous parameter combinations, which can be time-consuming and costly. Moreover, when it comes to mass manufacturing, generating parameters for each device adds significant production overhead. Alternatively, applying generic parameters may streamline the process but results in degraded control performance.

This technology eliminates these limitations by generating device-specific PID parameters in milliseconds with no tuning loops or device interaction. It leverages deep reinforcement learning to exploit complex relationships between device-specific information, multi-objective control requirements and PID parameters.

Technology Overview

University College London researchers used deep reinforcement learning techniques to find an optimized PID control policy. The policy is trained end-to-end on a small set of devices, enabling seamless deployment on new devices once the training phase is completed. Input requires only generic information about the device, typically known from manufacturing. A deep reinforcement learning agent generates PID parameters for new devices in milliseconds based on this information and a multi-objective control requirement, meeting product performance requirements.

The process doesn’t require any physical interaction with devices; the production implementation is entirely loop-free and offline.

The method has been successfully integrated into production at a market-leading optical communications technology company. The technology has significantly increased yield in manufacturing and improved product performance by more than 20%, demonstrating the practical benefits and added value that this advanced PID optimization approach brings to the industry.

Benefits

• Generates device-specific PID parameters in milliseconds.

• Requires no device interaction or tuning loops.

• Integrates seamlessly into any manufacturing pipeline.

• Already commercially proven to increase control performance and manufacturing yield in real production environments.

Applications

The technology could be used to improve PID control performance in various industrial automation sectors. Its applications extend to industries including:

• Automotives

• Chemical processing

• Robotics

• Machining/CNC

• Pharmaceutical manufacturing

• Water and wastewater treatment

• Heating, Ventilation and Air Conditioning

Opportunity

The team is seeking a partner for commercial and/or licensing opportunities that will enable them to expand into new markets.

IP Status

Patent application submitted