Skip to content

DataFrameColumn: Unable to Filter by Null via ElementWiseXYZ Methods (System.NotSupportedException) #6110

Closed

Description

@JD-Robbs

System Information (please complete the following information):

  • OS & Version: Windows 10
  • ML.NET Version: Latest
  • .NET Version: .NET 6.0

Describe the bug
When passing null into ElementWiseXYZ methods, a System.NotSupportedException is thrown: 'Specified method is not supported.'

To Reproduce
Steps to reproduce the behavior:

        var col = new DoubleDataFrameColumn("col", new double?[] {1.23, null, 2, 3 });
        var dfTest = new DataFrame(col);

        var filtered2Df = dfTest.Filter(dfTest["col"].ElementwiseNotEquals(2));
        var filteredNullDf = dfTest.Filter(dfTest["col"].ElementwiseNotEquals(null));

Expected behavior
I expected the resulting DataFrame stored in filteredNullDf to not include the value null - much like filered2Df does not include the value 2.

The only way of making it work is something like this:

         dfTest.Filter(new PrimitiveDataFrameColumn<bool>("filter", dfTest["col"].Cast<double?>().Select(v => v != null)));

Activity

  1. added this to the ML.NET Future milestone on Mar 8, 2022
  2. ghost added
    in-prA pull request is open to resolve this issue
    on Jun 6, 2023
  3. ghost removed
    in-prA pull request is open to resolve this issue
    on Jul 7, 2023
  4. ghost locked as resolved and limited conversation to collaborators on Aug 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions