Skip to content

Using numcodecs in V3 #2398

Description

@normanrz

We should allow users to set filters and compressor in zarr.open etc. for v3 arrays for backwards compatilibility.
The way I would do it is to mark both kwargs as deprecated while transforming the input into the new v3 metadata format. As codec implementations, the new numcodecs wrappers from zarr-developers/numcodecs#524 would be used.

As an example

from numcodecs import Blosc, Delta
filters = [Delta(dtype='i4')]
compressor = Blosc(cname='zstd', clevel=1, shuffle=Blosc.SHUFFLE)

would be turned into

"codecs": [
  {
    "name": "numcodecs.delta",
    "configuration": { "dtype": "i4" }
  }, {
    "name": "bytes",
    "configuration": { "endian": "little" }
  }, {
    "name": "blosc",
    "configuration": {
      "cname": "zstd",
      "clevel": 1,
      "shuffle": "shuffle",
      "typesize": 4,
      "blocksize": 0
  }
]

with the following warning

The use of `filters` and `compressor` is deprecated for Zarr version 3 arrays and may be removed in future releases of zarr-python. 
Your configuration has been transformed to match the new codec configuration. 
Please use the following codec configuration in the future:
codecs=[numcodecs.zarr3.Delta(dtype="i4"), zarr.codecs.Bytes(), zarr.codecs.Blosc(cname="zstd", clevel=1, shuffle="shuffle", typesize=4, blocksize=0)] 

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions