A multiresolution analysis-assisted reinforcement learning approach to run-by-run control

作者：Ganesan Rajesh^*; Das Tapas K; Ramachandran Kandethody M

来源：IEEE Transactions on Automation Science and Engineering, 2007, 4(2): 182-193.

DOI：10.1109/TASE.2006.879915

• chemical mechanical planarization (CMP)
• exponentially weighted moving average (EWMA)
• multiresolution
• reinforcement learning
• run-by-run control
• wavelet
• wavelet-modulated reinforcement learning-run-by run (WRL-RbR)

In recent years, the run-by run (RbR) control mechanism has emerged as a useful tool for keeping complex semiconductor manufacturing processes on target during repeated short production runs. Many types of RbR controllers exist in the literature of which the exponentially weighted moving average (EWMA) controller is widely used in the industry. However, EWMA controllers are known to have several limitations. For example, in the presence of multiscale disturbances and lack of accurate process models, the performance of EWMA controller deteriorates and often fails to control the process. Also, the control of complex manufacturing processes requires sensing of multiple parameters that may be spatially distributed. New control strategies that can successfully use spatially distributed sensor data are required. This paper presents a new multiresolution analysis (wavelet) assisted reinforcement learning (RL)-based control strategy that can effectively deal with both multiscale disturbances in processes and the lack of process models. The novel idea of a wavelet-aided RL-based controller represents a paradigm shift in the control of large-scale stochastic dynamic systems of which the control problem is a subset. Henceforth, we refer our new control strategy as a WRL-RbR controller. The WRL-RbR controller is tested on a multiple-input-multiple-output chemical mechanical planarization process of wafer fabrication for which the process model is available. Results show that the RL controller outperforms EWMA-based controllers for low autocorrelation. The new controller also performs quite well for strongly autocorrelated processes for which the EWMA controllers are known to fail. Convergence analysis of the new breed of the WRL-RbR controller is presented. Further enhancement of the controller to deal with model-free processes and for inputs coming from spatially distributed environments are also discussed.

• 出版日期2007-4