An Extension of Iterative Scaling for Decision and Data Aggregation in Ensemble Classification
Authors: Pal, Siddharth; Miller, David1
Source: The Journal of VLSI Signal Processing, Volume 48, Numbers 1-2, August 2007 , pp. 21-37(17)
Publisher: Springer
Key:
- Free Content
- New Content
- Subscribed Content
- Free Trial Content
- Free Content
- New Content
- Subscribed Content
- Free Trial Content
Abstract:
Improved iterative scaling (IIS) is an algorithm for learning maximum entropy (ME) joint and conditional probability models, consistent with specified constraints, that has found great utility in natural language processing and related applications. In most IIS work on classification, discrete-valued “feature functions” are considered, depending on the data observations and class label, with constraints measured based on frequency counts, taken over hard (0-1) training set instances. Here, we consider the case where the training (and test) set consist of instances of probability mass functions on the features, rather than hard feature values. IIS extends in a natural way for this case. This has applications (1) to ME classification on mixed discrete-continuous feature spaces and (2) to ME aggregation of soft classifier decisions in ensemble classification. Moreover, we combine these methods, yielding a method, with proved learning convergence, that jointly performs (soft) decision-level and feature-level fusion in making ensemble decisions. We demonstrate favorable comparisons against standard Adaboost.M1, input-dependent boosting, and other supervised combining methods, on data sets from the UC Irvine Machine Learning repository.Keywords: improved iterative scaling; maximum entropy; ensemble classification; mixed continuous-discrete feature spaces
Document Type: Research article
DOI: 10.1007/s11265-006-0009-6
Key:
- Free Content
- New Content
- Subscribed Content
- Free Trial Content
- Free Content
- New Content
- Subscribed Content
- Free Trial Content
Click here for Page Help