Repository navigation
Consider defaulting Ensemble Stacking to a trainer in StandardLearners #682
Description
See the conversation here: #681 (comment)
Ensemble Stacking defaults to using FastTree when users don't specify an underlying trainer. This results in a non-ideal dependency from Microsoft.ML.Ensemble to Microsoft.ML.FastTree, and would cause problems if we ever considered separating FastTree into its own NuGet package.
We should consider making a different default trainer under our Stacking use something in the StandardLearners assembly.
/cc @TomFinley
Activity
Zruty0 commentedon Aug 17, 2018
Another reason to do this is that FastTree is already an ensemble :) I see limited value in an ensemble of FastTrees.
TomFinley commentedon Aug 17, 2018
Hi @eerhardt . It is certainly undesirable that this dependency exists. On the other hand, if we wanted to use something in standard learners, that probably means something like a linear learner, and ensemble learning of linear learners is practically a bit less helpful as a concept.
My preference might be to simply not have a default at all: if someone wants to employ one of these meta-trainers, they have to tell us what underlying trainer they want to employ. It is a bit strange that this one decision, certainly by far the most consequential decision the user has to make when employing this method, has a default. But I know some people (not me) are allergic to to the idea that a trainer would require configuration.
Hi @Zruty0 , actually ensembles of FastTrees models have historically been quite good models. There are two very distinct types of ensembling going on with this configuration: boosting (where the trees are directly dependent on each other), and ensembling based on different samplings of the data (where each learnt model is less directly dependent)... that is, basically the difference between this and this. On the other hand we have the "bagging" functionality in FastTree itself to enable this, though I feel like the implementation there has some problems (not least that it due to various restrictions doesn't actually do bagging, despite its name).
Zruty0 commentedon Aug 17, 2018
Thanks for the explanation on ensembles of ensembles.
I think that should be true for the basic scenarios, like "just fit me a linear regression, what's so hard about it?", but I agree that this argument should not apply to meta-learners, as they are not to be considered basic scenarios.
Zruty0 commentedon Oct 29, 2018
@TomFinley , should we close this then?
TomFinley commentedon Nov 7, 2018
@Zruty0 , as it happens I have to fix this issue anyway for completely incidental reasons.
An elaborate series of changes that are, incredibly, actually related.
An elaborate series of changes that are, incredibly, actually related. (