Repository navigation
All DataFrame Elementwise methods uncorrectly work with NULL values #6820
Description
Describe the bug
PrimitiveColumn ElementwiseEquals, ElementwizeNotEquals and other Elementwise method ignore validity information
For example. this unit test fails as it ElementWizeEquals return true for the first item ( as 0 == null):
[Fact]
public void TestElementWiseEquals()
{
PrimitiveDataFrameColumn<int> intColumn1 = new PrimitiveDataFrameColumn<int>("Int1", new int?[] { 0, 2, 3 });
PrimitiveDataFrameColumn<int> intColumn2 = new PrimitiveDataFrameColumn<int>("Int2", new int?[] { null, 2, 3 });
var results = intColumn1.ElementwiseEquals(intColumn2);
Assert.False(results[0]);
}
Activity
rosebyte commentedon Sep 29, 2026
Triage: seems a bug, we should fix it, any help would be appreciated.
adestr commentedon Sep 30, 2026
I'm happy to take a look.
The underlying reason for this behaviour appears to be that the comparison operation acts on
ReadOnlyBuffer<T>, without first checking whether the buffer contains a valid data point.Simple fix seems to be something like:
Would need a careful eye on performance, of course. I'll investigate a bit further.
Add 857 contribution: dotnet/machinelearning#6820
Respect null bitmaps in primitive column comparisons (#7762)