Torchvision custom operator documentation - #6490
mdabek-nvidia wants to merge 2 commits into
Conversation
Signed-off-by: Marek Dabek <mdabek@nvidia.com>
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
Signed-off-by: Marek Dabek <mdabek@nvidia.com>
|
| "class TrimBorder(Operator):\n", | ||
| " preprocess_data = get_HWC_from_layout_pipeline\n", |
There was a problem hiding this comment.
Running the notebook reaches class TrimBorder(Operator) without defining or importing Operator, so the example immediately raises NameError. The later code also uses unimported get_HWC_from_layout_pipeline, dali, and fn, and calls _stack_int64, which has no definition in the notebook or repository. Add the required imports and use a defined helper such as the fn.stack pattern used by existing operators so readers can run the documented implementation.
| "@adjust_input\n", | ||
| "def crop(\n", | ||
| " inpt: TensorLike | ndd.Batch,\n", | ||
| " top: int | float,\n", | ||
| " left: int | float,\n", | ||
| " height: int | float,\n", | ||
| " width: int | float,\n", | ||
| " device: DeviceLike = \"cpu\",\n", | ||
| ") -> ndd.Tensor | ndd.Batch: ..." |
There was a problem hiding this comment.
Functional example returns None
This example uses names that the notebook does not import, and its ... body returns None. Once the imports are resolved, adjust_input unconditionally calls .evaluate() on that result, causing an AttributeError instead of applying a crop. Show and return the actual Dynamic Mode operation, as the existing crop implementation does with ndd.slice, so readers receive a usable example.
| "id": "498db99a", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "A minimal custom operator only needs to implement `_kernel`, which contains the transformation's logic. If the operator takes configuration parameters, implement `__init__` as well. To validate those parameters, provide `arg_rules` — either existing validators or custom ones deriving from `_ArgumentValidateRule`. If the input itself needs validation (for example, restricting the supported number of channels), list the corresponding rules in `input_rules`. Finally, if additional data processing is needed outside of the DALI pipeline, set `preprocess_data` to a processing function. When in doubt, check how the existing operators are implemented." |
There was a problem hiding this comment.
Guidance relies on private APIs
The chapter tells users to derive validators from _ArgumentValidateRule, but that underscore-prefixed class, _DataValidateRule, and Operator are not exported by the public Torchvision package. This makes custom operators depend on private implementation details that may change without compatibility guarantees. Either expose a supported extension API or avoid presenting these private classes as user-facing extension points.
| " \"\"\"\n", | ||
| " type(self).verify_data(data_input)\n", | ||
| "\n", | ||
| " # Original input is transfered to GPU, before being preprocess_data.\n", |
There was a problem hiding this comment.
The new comment spells “transferred” as “transfered.” The repository's fix-typos directive requires typos in documentation and comments to be corrected before merging.
| " # Original input is transfered to GPU, before being preprocess_data.\n", | |
| " # Original input is transferred to GPU, before being preprocess_data.\n", |
Rule Used: Fix typos in identifiers, comments, error messages, and docs. Common ones spotted in DALI history: sequenece, Allready, Whtether, milliseoncds, on-pair, Succesful, poitner, funciton, syncrhonized, deleteion, retruned, `worskpace... (source)
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
jantonguirao
left a comment
There was a problem hiding this comment.
The P1 items flagged by greptile should be fixed before merging:
- Missing imports / undefined
_stack_int64in theTrimBorderexample (line 936) - The
cropfunctional example returnsNone/Ellipsis, which breaksadjust_input(line 1076)
Category:
Other (Documentation)
Description:
Adding Implementing custom operator chapter to Torchvision documentation
Additional information:
Affected modules and functionalities:
Key points relevant for the review:
Check if new chapter could be clearly understood and if user is able to implement a custom operator based on this information.
Tests:
Checklist
Documentation
DALI team only
Requirements
REQ IDs: N/A
JIRA TASK: DALI-4809