Repository navigation
Non-standard naming in L-BFGS Learners (LogisticRegression, PoissonRegression) #2882
Description
In LogisticRegression and PoissonRegression (which use the same L-BFGS base), we have a parameter IterationsToRemember that refers to the number of gradients to accumulate in the history. While this terminology makes sense, it's not what we usually encounter in the field.
In the literature, we see this referred to as the "history size" (see e.g. wikipedia).
In the various toolkits that expose an L-BFGS solver, they use:
Scikit Learn: Doesn't expose it.
Spark: NumberOfCorrections
TensorFlow: num_correction_pairs
PyTorch: history_size
I would vote for HistorySize, with the docs explaining what it is.
What do you all think? Any big feelings around this?
Activity
rogancarr commentedon Mar 7, 2019
@shauheen @wschin @TomFinley @markusweimer What do you think?
[-]Non-standard naming in LogisticRegression[/-][+]Non-standard naming in L-BFGS Learners (LogisticRegression, PoissonRegression)[/+]Ivanidzo4ka commentedon Mar 7, 2019
Just to protect myself from pitchforks, before it was
MemorySize. :Drogancarr commentedon Mar 7, 2019
Oh, I don't think
IterationsToRememberis necessarily bad, just not standard :)TomFinley commentedon Mar 7, 2019
I don't remember that. 😉
Jokes aside, yes totally sounds fine with me. Memory size is a bad name since that could be misinterpreted. The idea of "corrections" is a bit weird; I mean, I guess if all the toolkits call it that, it's just weird to name it something so different from the literature which I seem to recall just being history.
I would also be fine with leaving it as is. I can't say I consider "correction size" fundamentally easier to remember than "iterations to remember," considering how L-BFGS uses that parameter. If anything that makes it more obscure, but I can't say I care very much.
rogancarr commentedon Mar 7, 2019
@TomFinley
HistorySizeorIterationsToRemember?shauheen commentedon Mar 7, 2019
HistorySize+1Enable xml docs for Data.Analysis (dotnet#2882)
Move DataFrame to machinelearning (#5641)
Merge from main repository (#1)
Actually merge from main (#2)
Update editor config and format samples (#5893)