Fault-Aware Hybrid MPC–Reinforcement Learning Framework for Adaptive Sensor Bias Handling in Industrial Furnace Temperature Control

MPC–Reinforcement Learning Framework for Furnace Temperature Control

Abstract

Temperature control in industrial furnaces is an important issue in view of product quality and efficiency. In the presence of sensor bias faults, the performance of the controller declines due to the use of incorrect feedback data. This paper presents a hybrid system of Model Predictive Control (MPC) combined with Reinforcement Learning (RL) for the detection and compensation of sensor bias faults in the control system of the furnace temperature. It uses MPC to achieve stable tracking performance as well as the creation of residuals, which indicate sensor abnormalities. It also includes a supervisory Deep Q-Network (DQN) agent, which chooses the best tuning strategy of MPC from the set of available options on the basis of tracking and residuals. The proposed methodology was tested on a simulated system in MATLAB/Simulink with the use of constant, periodic, and multilevel reference temperature profiles with both negative and positive sensor bias. The training procedure took 57 episodes in the case of negative bias and 77 episodes in the absence of any faults when a constant reference is provided. Other types of the reference temperature profile were more difficult to be learned by the DQN agent. They took 120 episodes without achieving convergence.