Skip to content

New API for ML.NET #754

New issue
New issue
Closed
Closed
New API for ML.NET#754

Description

@Zruty0
Zruty0
opened on Aug 27, 2018

We are creating a stable API that:

  • Uses parallel terminology with other well-known ML libraries (Spark, sklearn);
  • Takes advantage of strong types of .NET to shorten path to success;
  • Is going to be present from now till 1.0 and beyond;
  • Keeps simple ML scenarios concise;
  • Allows advanced ML scenarios: see Direct API: Scenarios to light up for V1 #584 .

To that end, we are going to expose a selection of Estimators and Transformers (see #581 ) that cover existing transforms, learners and loaders.

This issue will be used to track the overall project status: what is planned to be done, what is done, etc.

Activity

  1. Zruty0
    added
    api
    on Aug 27, 2018

  2. Zruty0 commented on Aug 27, 2018

    @Zruty0
    Zruty0
    on Aug 27, 2018
    ContributorAuthor

    List of transforms:

    Transform Category Priority Has entry point Status
    Key To Value Transform Categorical 0 1 #856
    Concat Transform Schema manipulation 0 1 #896
    Term Transform Categorical 0 1 #759
    Text Transform Text processing 0 1 #801
    Image Greyscale Transform Image 0 0 #753
    Image Loader Transform Image 0 0 #753
    Image Pixel Extractor Transform Image 0 0 #753
    Image Resizer Transform Image 0 0 #753
    Tensorflow Scoring Transform Image 0 1 #840 #877
    Min-Max Normalizer Normalizer 0 1 #797
    NA Handle Transform Missing values 1 1 no need
    NA Indicator Transform Missing values 1 1 #1217
    NA Replace Transform Missing values 1 1 #917
    Categorical Transform Categorical 1 1 #899
    Categorical Hash Transform Categorical 1 1 #1033
    Hash Transform Categorical 1 0 #944
    Copy Columns Transform Schema manipulation 1 1 #706
    Word Embeddings Transform Text processing 1 1 #928
    Key To Vector Transform Categorical 1 0 #858
    Character Tokenizer Transform Text processing 2 1 #931
    Ngram Hash Transform Text processing 2 0 #953
    Ngram Transform Text processing 2 1 #953
    Stopwords Remover Transform Text processing 2 0 #953
    Text Normalizer Transform Text processing 2 0 #953
    Word Bag Transform Text processing 2 0 #953
    Word Hash Bag Transform Text processing 2 0 #953
    Word Tokenizer Transform Text processing 2 1 #931
    Convert Transform Column mapper 2 1
    Drop Slots Transform Feature selection 2 0 no need
    LogMeanVar Normalizer Normalizer 2 1 #797
    MeanVar Normalizer Normalizer 2 1 #797
    Latent Dirichlet Allocation Transform Projection 2 1 #972
    Tree Ensemble Featurization Transform Projection 2 1
    NA Filter Row manipulation 2 1
    Count Feature Selection Transform Feature selection 2 1 #991
    Mutual Information Feature Selection Transform Feature selection 2 1 #991
    NA Drop Transform Missing values 2 1
    Binning Normalizer Normalizer 2 1 #797
    Global Contrast Normalization Transform Normalizer 2 1 #961
    Principal Component Analysis Transform Projection 2 1 #1333
    Random Fourier Features Transform Projection 2 0 #1122
    Bootstrap Sample Transform Row manipulation 2 1
    Shuffle Transform Row manipulation 2 0
    Custom Stopwords Remover Transform Text processing 2 0
    Learner Feature Selection Transform Feature selection 3 0
    Lp-Norm Normalizer Normalizer 3 1 #961
    Supervised Binning Normalizer Normalizer 3 1
    Whitening Transform Normalizer 3 0 #961, #1326
    Choose Columns Transform Schema manipulation 3 0
    Drop Columns Transform Schema manipulation 3 1
    Generate Number Transform Schema manipulation 3 1
    Keep Columns Transform Schema manipulation 3 0
    Key To Binary Vector Transform Categorical 3 0 #858
    Term Lookup Transform Categorical 3 0
    Label Indicator Transform Column mapper 3 1
    Group Transform Relational operation 3 1
    Un-group Transform Relational operation 3 1
    Range Filter Row manipulation 3 1
    Skip Filter Row manipulation 3 1
    Skip and Take Filter Row manipulation 3 1
    Take Filter Row manipulation 3 1
    Evaluate Predictor Re-evaluate 4 0
    Hash Join Transform Re-evaluate 4 1
    Load Transform Re-evaluate 4 0
    Optional Column Transform Re-evaluate 4 1
    Score Predictor Re-evaluate 4 0
    Sentiment Analyzing Transform Re-evaluate 4 1
    Train and Score Predictor Re-evaluate 4 0

  3. Zruty0 commented on Aug 27, 2018

    @Zruty0
    Zruty0
    on Aug 27, 2018
    ContributorAuthor

    List of trainers:

    Trainer Category Priority Status
    SDCA: Fast Linear (SA-SDCA) Linear 0 #716
    SDCAMC: Fast Linear Multi-class Classification (SA-SDCA) Linear 0 #716
    SDCAR: Fast Linear Regression (SA-SDCA) Linear 0 #716
    AveragedPerceptron: Averaged Perceptron Linear 0 #849
    OVA: One-vs-All Meta 0 #865
    FieldAwareFactorizationMachine: Field-aware Factorization Machine FFM 0 #912
    FastTreeBinaryClassification: FastTree (Boosted Trees) Classification Tree 0 #855
    FastTreeRegression: FastTree (Boosted Trees) Regression Tree 0 #855
    KMeansPlusPlus: KMeans++ Clustering Clustering 0 #979
    LightGBMBinary: LightGBM Binary Classifier Tree 0 #962
    LightGBMMulticlass: LightGBM Multi-class Classifier Tree 0 #962
    LightGBMRegression: LightGBM Regressor Tree 0 #962
    LogisticRegression: Logistic Regression Linear 0 #957
    MultiClassLogisticRegression: Multi-class Logistic Regression Linear 0 #957
    OLSLinearRegression: Ordinary Least Squares (Regression) Linear 0 #1002
    SymbolicSGD: Symbolic SGD (binary) Linear 0 #1012
    FastForestClassification: Fast Forest Classification Tree 0 #855
    FastForestRegression: Fast Forest Regression Tree 0 #855
    BinaryClassificationGamTrainer: Generalized Additive Model for Binary Classification GAM 1 #962
    RegressionGamTrainer: Generalized Additive Model for Regression GAM 1 #962
    OnlineGradientDescent: Stochastic Gradient Descent (Regression) Linear 1 #849
    PoissonRegression: Poisson Regression Linear 1 #957
    PKPD: Pairwise coupling (PKPD) Meta 1 #865
    pcaAnomaly: PCA Anomaly Detector Projection 1 #996
    FastTreeTweedieRegression: FastTree (Boosted Trees) Tweedie Regression Tree 1 #855
    PriorPredictor: Prior Predictor Baseline 2 #875
    RandomPredictor: Random Predictor Baseline 2 #875
    MultiClassNaiveBayes: Multiclass Naive Bayes Bayes 2 #1111
    BinarySGD: Hogwild SGD (binary) Linear 2 #1134
    LinearSVM: SVM (Pegasos-Linear) Linear 2 #849
    EnsembleRegression: Regression Ensemble (bagging, stacking, etc) Meta 2 On hold until we decide how to change the ensemble learners #1152
    WeightedEnsemble: Parallel Ensemble (bagging, stacking, etc) Meta 2 On hold until we decide how to change the ensemble learners #1152
    WeightedEnsembleMulticlass: Multi-class Parallel Ensemble (bagging, stacking, etc) Meta 2 On hold until we decide how to change the ensemble learners #1152
    FastTreeRanking: FastTree (Boosted Trees) Ranking Tree 2 #855
    LightGBMRanking: LightGBM Ranking Tree 2 #962
  4. singlis
    added a commit that references this issue on Oct 15, 2018

    Replace DropColumns,KeepColumns and ChooseColumns with SelectColumns

    1b0fff9
  5. shauheen
    modified the milestone: 1018 on Oct 16, 2018
  6. singlis
    added 3 commits that reference this issue on Oct 20, 2018

    Replace DropColumns,KeepColumns and ChooseColumns with SelectColumns (#…

    9a33cd4

    Removes ChooseColumnsTransform and DropColumnsTransform classes

    c8b5a56

    Replaces ChooseColumnsTransform and DropColumnsTransform with SelectC…

    c2b5e76
  7. singlis
    added a commit that references this issue on Nov 21, 2018

    Addition of the ValueMappingEstimator and ValueMappingTransform.

    36ffe1c
  8. sfilipi
    added a commit that references this issue on Dec 21, 2018

    Addition of the ValueMappingEstimator and ValueMappingTransform. (#1710)

    06ab3d0

  9. shauheen commented on Jan 8, 2019

    @shauheen
    shauheen
    on Jan 8, 2019
    Contributor

    @Ivanidzo4ka can you please verify if this can be closed now?

  10. singlis
    added a commit that references this issue on Jan 24, 2019

    ValueMappingEstimator example

    bfdbc4f
  11. singlis
    added a commit that references this issue on Feb 1, 2019

    ValueMappingEstimator example (#2222)

    578c188

  12. eerhardt commented on Feb 22, 2019

    @eerhardt
    eerhardt
    on Feb 22, 2019
    Member

    I believe this can be closed. We have made this new API and have separate issues tracking remaining necessary API work.

    Please re-open if I'm wrong.

  13. eerhardt
    closed this as completedon Feb 22, 2019
  14. ghost locked as resolved and limited conversation to collaborators on Mar 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions