Options

# AdaBoost vs. BayesianBoosting

Hi!

I'm goint o try some boosting for my bachelor's thesis. I haven't yet decided whether using AdaBoost or BayesianBoosting. Actually I don't understand all the differences. What I do understand is that BayesianBoosing can use different fractions of the example set for model fitting and performance estimation. I understand that it is able to reweight examples to ensure equally distributed labels. But what exactly means [tt]allow_marginal_skews[/tt]?

Martin Scholz (the author of the operator) cites in the help text Scholz/2005b. Can anyone give me publication details on (t)his work. I think I would understand the differencen in detail if i read it.

Best regards,

chero

I'm goint o try some boosting for my bachelor's thesis. I haven't yet decided whether using AdaBoost or BayesianBoosting. Actually I don't understand all the differences. What I do understand is that BayesianBoosing can use different fractions of the example set for model fitting and performance estimation. I understand that it is able to reweight examples to ensure equally distributed labels. But what exactly means [tt]allow_marginal_skews[/tt]?

Martin Scholz (the author of the operator) cites in the help text Scholz/2005b. Can anyone give me publication details on (t)his work. I think I would understand the differencen in detail if i read it.

Best regards,

chero

Tagged:

0

## Answers

537MavenBayesian Boosting is like creating all possible models and weighting them according their accuracy?

(Maybe you also need to weigh them according the model prior).

Most of the time this is not feasible in practice.

Adaboost, adaptive boosting, setting the iteration parameter to n, creates n models.

Models are weighted by their accuracy.

New training data is created by reweighing examples.

If examples are correctly classified in the previous iteration their weight goes down,

if examples are incorrectly classified their weight goes up.

Freund and Shaffire prove that the error on the training set goes down exponentially fast, using adaboost.

This paper is really good?

vorlon.case.edu/~sray/eecs600_fall08/ensembles_survey.ps

Ensemble Methods in Machine Learning Dietterich filetype:pdf

1,751RM Founderyou can find Martin's publication here:

http://www-ai.cs.uni-dortmund.de/auto?self=$Publication_e9zx9gcx

Here are his other publications on this and related topics:

http://www-ai.cs.uni-dortmund.de/PERSONAL/scholz.html

Cheers,

Ingo

537MavenNot about ensembles or boosting.

1,751RM FounderCheers,

Ingo

53MavenThanks a lot for the links - was searching the same just over the weekend :-)

would propose to add these references to the wiki. There is a general lack of documentation where the various algorithms come from... so the wiki would be the natural place growing such a knowledge library.

Problem is that it appears that the wiki wants to be a duplication of the built-in help. If this is so, you'll end up in a maintenance problem as I expect there is only a one-way conversion path?

Stefan

1,751RM Founderhttp://rapid-i.com/rapidforum/index.php/topic,2013.msg8148.html#msg8148

But I must admit that I am not sure about the current state of this.

Cheers,

Ingo