A gradient method for unconstrained optimization in noisy environment

作者：Krejic Natasa; Luzanin Zorana^*; Stojkovska Irena

来源：Applied Numerical Mathematics, 2013, 70: 1-21.

DOI：10.1016/j.apnum.2013.02.006

• Stochastic optimization
• Stochastic approximation
• Noisy function
• Gradient method
• Line-search method

A gradient method for solving unconstrained minimization problems in noisy environment is proposed and analyzed. The method combines line-search technique with Stochastic Approximation (SA) method. A line-search along the negative gradient direction is applied while the iterates are far away from the solution and upon reaching some neighborhood of the solution the method switches to SA rule. The main issue is to determine the switching point and that is resolved both theoretically and practically. The main result is the almost sure convergence of the proposed method due to a finite number of line-search steps followed by infinitely many SA consecutive steps. The numerical results obtained on a set of standard test problems confirm theoretical expectations and demonstrate the efficiency of the method.

• 出版日期2013-8