A novel approach to optimize the objective function based on VC dimension and structural risk minimization

作者：Qiu Xintao^*; Fu Dongmei; Yang Tao

来源：30th Chinese Control Conference, CCC 2011, 2011-07-22 to 2011-07-24.

• Algorithm efficiency
• Empirical risks
• Generalization performance
• Learning machines
• Learning methods
• Learning problem
• Objective functions
• Optimization goals
• Optimization principle
• Over fitting problem
• Practical method
• Structural risk minimization
• VC dimension

So far, lots of algorithms and learning methods are taking the empirical risk for the optimization goal. In this paper, we propose a new way to optimize the objective function based on VC dimension and structural risk minimization. The optimized function F is firstly defined by us, and some effective design forms of it will also be given. Then we fulfill a useful criterionthe balance minimum optimization principle. This principle considers not only the empirical risk, but also considers the VC dimension of learning machine. Balancing the two factors can avoid the underfitting problem and overfitting problem. Experimental results show that the method we proposed is effective to improve the property of algorithm efficiency, convergence and generalization of the learning machine. Also, the proposed principle for optimization is a new criterion, which is not a practical method for a particular problem of improvement. Therefore, this method is suitable to be applied in many practical situations, which may bring a good generalization performance and efficiency in some learning problems.

• 出版日期2011
• 单位北京科技大学