Increasing the Robustness of Boosting Algorithms within the Linear-programming Framework

Authors: Sun, Yijun¹; Todorovic, Sinisa²; Li, Jian

Source: The Journal of VLSI Signal Processing, Volume 48, Numbers 1-2, August 2007 , pp. 5-20(16)

Publisher: Springer

Buy & download fulltext article:

Price: $47.00 plus tax (Refund Policy)

Abstract:

AdaBoost has been successfully used in many signal classification systems. However, it has been observed that on highly noisy data AdaBoost easily leads to overfitting, which seriously constrains its applicability. In this paper, we address this problem by proposing a new regularized boosting algorithm LP_norm2-AdaBoost (LPNA). This algorithm arises from a close connection between AdaBoost and linear programming. In the algorithm, skewness of the data distribution is controlled during the training to prevent outliers from spoiling decision boundaries. To this end, a smooth convex penalty function (l ₂ norm) is introduced in the objective function of a minimax problem. A stabilized column generation technique is used to transform the optimization problem into a simple linear programming problem. The effectiveness of the proposed algorithm is demonstrated through experiments on many diverse datasets.

Keywords: pattern classification; large margin classifier; AdaBoost; linear programming; minimax problem; soft margin; regularization

Document Type: Research article

DOI: http://dx.doi.org/10.1007/s11265-006-0006-9

Affiliations: 1: Email: sun@dsp.ufl.edu 2: Email: sintod@vision.ai.uiuc.edu

Publication date: 2007-08-01