Skip to content

LightGbm default evaluation metrics in ML.NET do not conform to standalone LightGbm #3822

Closed

Description

@najeeb-kazmi

Found while investigating fix for #3761

In ML.NET LightGbm wrapper, the default EvaluationMetric is set to EvaluateMetricType.Error for multiclass, EvaluationMetricType.LogLoss for binary, and so on. On the other hand, in standalone LightGbm, the default evaluation metric is "", which means that LightGbm will automatically select the default metric for the given objective function.

This leads to inconsistent behavior from the user's perspective: If a user specified EvaluationMetric = EvaluateMetricType.Default, the parameter passed to LightGbm would be the empty string "" but if they do not specify EvaluationMetric at all, the parameter passed to LightGbm would be Error for multiclass, LogLoss for binary, and so on.

We need to investigate whether these metrics are indeed the defaults for the respective objective functions in LightGbm, and if they are not, then change the defaults in ML.NET to conform to standalone LightGbm. Note that this would be a breaking change.

Activity

  1. self-assigned this
    on Jun 4, 2019
  2. added a commit that references this issue on Jun 4, 2019
  3. ghost locked as resolved and limited conversation to collaborators on Mar 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions