Skip to content

Prevent silent integer overflows in arithmetic operators - #6481

Open
rostan-t wants to merge 4 commits into
NVIDIA:mainfrom
rostan-t:fix-arithmetic-int-overflows
Open

rostan-t wants to merge 4 commits into
NVIDIA:mainfrom
rostan-t:fix-arithmetic-int-overflows

Conversation

@rostan-t

@rostan-t rostan-t commented Sep 11, 2026 •

Copy link
Copy Markdown
Collaborator

Category:

Bug fix (non-breaking change which fixes an issue)

Description:

Currently, arithmetic operators pull constant integers as 32 bits signed ints. As a result, silent overflows can happen, producing wrong results without any warning nor error.

The Python binding forOpSpec::AddArg is already registered as int64_t:

DALI_OPSPEC_ADDARG(int64_t)

And the argument storage uses int64_t for all integral types anyway:

template <typename T>
struct argument_storage {
using type = std::conditional_t<
std::is_integral<T>::value || std::is_enum<T>::value,
int64_t, T>;
};

So we can pull the integer constants as 64 bit integers and error out if they don't fit their target dtype.

Additional information:

This allows arithmetic operators to consume more than 32 bit signed integers but throws an exception instead of silently overflowing.

Affected modules and functionalities:

Integral constants in arithmetic operators.

Key points relevant for the review:

Tests:

  • Existing tests apply
  • New tests added
    • Python tests
    • GTests
    • Benchmark
    • Other
  • N/A

Checklist

Documentation

  • Existing documentation applies
  • Documentation updated
    • Docstring
    • Doxygen
    • RST
    • Jupyter
    • Other
  • N/A

DALI team only

Requirements

  • Implements new requirements
  • Affects existing requirements
  • N/A

REQ IDs: N/A

JIRA TASK: N/A

@greptile-apps

greptile-apps Bot commented Sep 11, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

The PR appears safe to merge, with no outstanding correctness or repository-rule findings in the scoring set.

Summary

This PR prevents silent narrowing of integral constants in arithmetic expressions by transporting them as 64-bit integers and validating them against their explicitly selected destination types.

  • Changes arithmetic-expression integer argument storage from 32-bit to 64-bit.
  • Rejects out-of-range integral constants with an OverflowError that preserves operator context.
  • Documents the transport and promotion behavior for Python constants.
  • Adds boundary and overflow-rejection tests for signed and unsigned constants.
Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Python integral constant] --> B[OpSpec integer_constants as int64]
  B --> C[ConstantStorage]
  C --> D{Fits selected dtype?}
  D -->|Yes| E[Cast and store constant]
  D -->|No| F[Throw OverflowError with operator context]
Loading

Reviews (5) · Last reviewed commit: "Update documentation to reflect that ari..."

@rostan-t

Copy link
Copy Markdown
Collaborator Author

!build

@dali-automaton

Copy link
Copy Markdown
Collaborator

CI MESSAGE: [67521712]: BUILD STARTED

@dali-automaton

Copy link
Copy Markdown
Collaborator

CI MESSAGE: [67521712]: BUILD FAILED

@dali-automaton

Copy link
Copy Markdown
Collaborator

CI MESSAGE: [67521712]: BUILD PASSED

Comment thread dali/operators/math/expressions/constant_storage.h Outdated
Comment thread dali/test/python/operator_1/test_arithmetic_ops.py Outdated
@review-notebook-app

Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

Comment thread dali/operators/math/expressions/constant_storage.h Outdated
Comment thread dali/test/python/operator_1/test_arithmetic_ops.py Outdated
Comment thread docs/examples/general/expressions/expr_type_promotions.ipynb Outdated
@rostan-t
rostan-t force-pushed the fix-arithmetic-int-overflows branch from cb01308 to 23a81ca Compare September 14, 2026 14:42
@rostan-t

Copy link
Copy Markdown
Collaborator Author

!build

@dali-automaton

Copy link
Copy Markdown
Collaborator

CI MESSAGE: [67797265]: BUILD STARTED

@dali-automaton

Copy link
Copy Markdown
Collaborator

CI MESSAGE: [67797265]: BUILD FAILED

Comment thread dali/test/python/operator_1/test_arithmetic_ops.py Outdated
@rostan-t

Copy link
Copy Markdown
Collaborator Author

!build

@dali-automaton

Copy link
Copy Markdown
Collaborator

CI MESSAGE: [67967537]: BUILD STARTED

@dali-automaton

Copy link
Copy Markdown
Collaborator

CI MESSAGE: [67967537]: BUILD PASSED

@mzient mzient self-assigned this Sep 16, 2026
Comment on lines +55 to +56
? spec.GetRepeatedArgument<int64_t>("integer_constants")
: std::vector<int64_t>{};

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this works by accident (kind of) - please also change the argument type in arithmetic.cc to DALI_INT64_VEC after #6486 is merged.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Signed-off-by: Rostan Tabet <rtabet@nvidia.com>
Signed-off-by: Rostan Tabet <rtabet@nvidia.com>
Signed-off-by: Rostan Tabet <rtabet@nvidia.com>
…e than int32

Signed-off-by: Rostan Tabet <rtabet@nvidia.com>
@rostan-t
rostan-t force-pushed the fix-arithmetic-int-overflows branch from 8281414 to ecdd524 Compare September 16, 2026 17:51
@rostan-t

Copy link
Copy Markdown
Collaborator Author

!build

@dali-automaton

Copy link
Copy Markdown
Collaborator

CI MESSAGE: [68224735]: BUILD STARTED

@dali-automaton

Copy link
Copy Markdown
Collaborator

CI MESSAGE: [68224735]: BUILD PASSED

@rostan-t
rostan-t requested a review from mzient September 17, 2026 17:00

@jantonguirao jantonguirao left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Confirmed the earlier review concerns (int64 storage, overflow check semantics, UINT32 boundary test fix, CI failure) are all resolved, and CI is green.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants