favorite3This error bound interpretation can open the door to new modifications of AdaBoost based, for example, on tuning the global and past asymmetry contributions in order to achieve different asymmetric behaviors along rounds.
favorite3In fact, weights initialization is no more than a way to modify the data distribution seen by the learner and, as we will see, it can be easily shown to shape the error bound that sets AdaBoost minimization goal.
favorite14This analysis suggests that, only with an unbalanced class-conditional initialization of the weight distribution, AdaBoost is, by itself, a theoretically sound asymmetric classification algorithm.
favorite5Eventhough some studies (Freund and Schapire, 1997; Zadrozny et al., 2003) mention that the incorporation of unbalanced initial weights could lead to a cost-sensitive version of AdaBoost, subsequent works insist that this is not enough to reach effective asymmetry (Viola and Jones, 2002; Mease et al., 2007; Sun et al., 2007; Masnadi-Shirazi and Vasconcelos, 2007) swelling the number of different asymmetric boosting algorithm variants.
favorite5Most of the other proposed algorithms (Karakoulas and Shawe-Taylor, 1998; Fan et al., 1999; Ting, 2000; Viola and Jones, 2002; Sun et al., 2007) try to reach asymmetry based on direct manipulations of the weight distribution update rule.
favorite7Classification Performance As we have just commented, though theoretically equivalent, classifiers obtained from AdaBoostDB and Cost-Sensitive AdaBoost tend to differ due to numerical errors related to the different model (polynomial vs.
favorite1Table 1: AdaBoostDB asymmetric behavior (false negatives, false positives, classification error and normalized expected cost) for each cost combination over the synthetic and UCI datasets.
favorite3AdaboostDB performance follows the trend set by the Bayes optimal classifier, describing a consistent and gradual asymmetric behavior across the different costs and all the studied parameters (false positives, false negatives, classification error and normalized expected cost).
favorite5Experiments To show and assess the performance of AdaBoostDB in practical terms we have conducted a series of empirical experiments to analyze the asymmetric behavior of the algorithm, comparing it with theoretical optimal classifiers and with Cost-Sensitive AdaBoost using synthetic and real datasets.
favorite3Abstract Based on the use of different exponential bases to define class-dependent error bounds, a new and highly efficient asymmetric boosting scheme, coined as AdaBoostDB (Double-Base), is proposed.
favorite18Figure 1c shows how, even in a scenario like the one proposed by Viola and Jones , the classifier obtained by AdaBoost after an asymmetric weight initialization follows a real cost-sensitive iterative profile.
favorite19In , following a different insight to analyze AdaBoost and obtaining a novel error bound interpretation, asymmetric weight initialization is shown to be an effective way to reach cost-sensitiveness, and, as occurs with everything related to boosting, it is achieved in an additive round-by-round (asymptotic) way.
favorite6Besides the great success of the detection framework, the authors themselves acknowledge that neither this a posteriori cost-sensitive tuning ensures that the selected weak classifiers are optimal for the asymmetric goal , nor their modifications preserve the original AdaBoost training and generalization guarantees .
favorite11It is also important to emphasize that this work is focused on AdaBoost and its cost-sensitive variants, a realm of methods in the literature that are based on a exponential loss minimization criterion, analogous to that giving rise to the original algorithm (as we have seen in Sections 2.1 and 2.2 from different points of view) .
favorite0Schapire and Yoram Singer proposed , from the original derivation of AdaBoost, a generalised and simplified analysis that models the algorithm as an additive (round-by-round) minimization process of an exponential bound on the strong classifier training error (ET ).
favorite2Figure 7: Classification asymmetry comparative between Cost-Generalized AdaBoost and AdaC1, across training rounds and costs, for the CBCL dataset..
favorite2Comparison of the Two Best Solutions As a result of our empirical analysis, two algorithms stand above the rest: AdaC1 and Cost-Generalized AdaBoost, with performance figures that are consistently better than those shown by the other alternatives in virtually all the tested scenarios.
favorite2Following our experimental framework, we have gathered the values of Classification Asymmetry obtained by Cost-Sensitive AdaBoost and Cost-Generalized AdaBoost after every training round of every dataset and fold, and for every cost combination.
favorite9Now, our empirical results not only corroborate that the obtained classifiers for those algorithms are different, but also that Cost-Generalized AdaBoost outperforms Cost-Sensitive AdaBoost in all the scenarios (Figure 2, Table 3 and Appendix), with a difference that is more marked the greater the asymmetry (Figure 4).
favorite6Moreover, the error bound defined by Cost-Generalized AdaBoost, able to preserve the class-dependent loss ratio regardless of the training round, seems to be more consistent than that used by the other theoretical alternatives, Cost-Sensitive AdaBoost and AdaBoostDB, showing a tendency to increasingly emphasize the least costly class (asymmetry swapping).