Approximation with neural networks activated by ramp sigmoids

作者：Cheang Gerald H L^*

来源：Journal of Approximation Theory, 2010, 162(8): 1450-1465.

DOI：10.1016/j.jat.2010.03.004

• Two-hidden-layer neural net approximation
• Ramp sigmoids
• Approximation of ellipsoids
• Approximation error bounds

Accurate and parsimonious approximations for indicator functions of d-dimensional balls and related functions are given using level sets associated with the thresholding of a linear combination of ramp sigmoid activation functions. In neural network terminology, we are using a single-hidden-layer perceptron network implementing the ramp sigmoid activation function to approximate the indicator of a ball. In order to have a relative accuracy epsilon, we use T = c(d(2)/epsilon(2)) ramp sigmoids, a result comparable to that of Cheang and Barron (2000) [4], where unit step activation functions are used instead. The result is then applied to functions that have variation V(f) with respect to a class of ellipsoids. Two-hidden-layer feedforward neural nets with ramp sigmoid activation functions are used to approximate such functions. The approximation error is shown to be bounded by a constant times V(f)/T(1)(1/2) + V(f)d/T(2)(1/1), where T(1) is the number of nodes in the outer layer and T(1) is the number of nodes in the inner layer of the approximation f(T1),(T2).

• 出版日期2010-8