(fix): structured arrays for v2 - #2681
Conversation
|
Hmm, this will need to handle the case where the array is not given the |
|
(P.S I used |
dstansby
added
the
needs release notes
github-actions
Bot
removed
the
needs release notes
| # In the case of zarr v2, the simplest i.e., '|VXX' dtype is represented as a string | ||
| dtype_descr = self.dtype.descr | ||
| if self.dtype.kind == "V" and dtype_descr[0][0] != "" and len(dtype_descr) != 0: | ||
| dtype_json = tuple(self.dtype.descr) | ||
| else: | ||
| dtype_json = self.dtype.str |
There was a problem hiding this comment.
This is my attempt to match the old behavior. I didn't look back at the old code yet, but if someone knows this to be wrong, would be great to know.
martindurant
left a comment
martindurant
left a comment
There was a problem hiding this comment.
I am keen to see this go in
|
|
||
|
|
||
| def parse_dtype(data: npt.DTypeLike) -> np.dtype[Any]: | ||
| if isinstance(data, list): # this is a valid _VoidDTypeLike check |
There was a problem hiding this comment.
This is to handle the [(field_name, field_dtype, field_shape), ...] case on https://numpy.org/doc/2.1/reference/arrays.dtypes.html#specifying-and-constructing-data-types but at the same time to obey
This might require more stringent checking or tests...Not sure. The reason this tuple conversion happens is that lists (as data types) incoming from on-disk reads contain lists, not tuples. So maybe we should check list and data[0] is also list? And throw an error if it isn't? I'm not sure what else could be in the lists though
There was a problem hiding this comment.
I guess the dtype constructor would make an exception (or our own comprehension fails) in the case the JSON on disk was edited - so I'm not too worried.
…ython into ig/structured_arrays_v2
|
|
||
|
|
||
| def parse_dtype(data: npt.DTypeLike) -> np.dtype[Any]: | ||
| if isinstance(data, list): # this is a valid _VoidDTypeLike check |
There was a problem hiding this comment.
I guess the dtype constructor would make an exception (or our own comprehension fails) in the case the JSON on disk was edited - so I'm not too worried.
|
great, thanks for working on this @ilan-gold and @martindurant |
4 participants
This is a best guess based on https://numpy.org/doc/2.1/reference/generated/numpy.dtype.kind.html and the fact that
VLenBytesappears to be explicitly for strings.This addresses the v2 case of #2134
TODO: