RFC-8: Add comment 1 (B. Best)

[btbest]

Hi OME-Team!

I've written up my comments on RFC-8.

I'm on holiday for a couple weeks now, so I wanted to get this up before.

I would go over this another time when I'm back, mainly to reduce the LLM vibe. I needed to speed up the text structuring a bit in the interest of time spent :) Hence marking this PR as draft.

If you want to push the first round of comments out while I'm out, I'm ok with it being merged as-is (the phrasing might not necessarily be my style everywhere, but the points are all there...). Feel free to modify the PR however you see fit otherwise.

[github-actions]

Automated Review URLs

• Readthedocs

[jo-mueller]

Hi @btbest , thanks for the in-depth feedback and the constructive comments.

Side-note about the confusion around the coordinate transformations between multiscale and singlescale, which has been largely added by myself:

At the time of writing, there were essentially two boundary conditions:

• Each singlescale should know its own scale
• Each multiscale should provide a single coordinate system which in 0.6 was called "intrisinc coordinate system" as a hook for other transforms

This leaves two possible ways of describing this relationship:

• The output of the transformation in the singlescale node points up:

  "output": {"path": "../", "id": "intrinsic"}

  Which has the drawback of making the singlescale non-portable, because copying it somewhere else will let the output reference go into the void
• Each singlescale could have its own coordinate system, and identity transformations link it to the multiscale's "intrinsic" coordinate system, which comes with the drawback of a transform graph consisting of n+1 coordinate systems and n*2 coordinate transformations for n resolution levels.

This is not to excuse that the section around the transforms isn't as tight/internally consistent as it should be, but just to explain the source of the complexity that led to the inconsistencies.

[jo-mueller]

Not sure this was only a thing in 0.6. The section on label attributes has been left pretty much unchanged from the previous versions (i.e., here in 0.5)

[btbest]

True, my mistake. Idk how I managed to read over the "Additional keys under colors are allowed." in 0.5 even though I could swear I specifically looked for it 😅 and on top of that it ended up reiterated multiple times in this section, what nonsense. I'll remove it when I get back

The point remains: color customisation separate from other label properties is an explicit capability of older versions and the current proposal quietly removes it.

I don't mind removing it, I assume it's not a particularly popular feature, but there should be a clearly stated migration path. I've already implemented label attributes in clearscale assuming color properties won't exist in the future. They're kept as regular label properties whose key is prefixed with "@color:" instead :)

[jo-mueller]

Again, not sure where in 0.6 this was specifically introduced? 🤔

[btbest]

Side-note about the confusion around the coordinate transformations between multiscale and singlescale, which has been largely added by myself:

How could you. I will forevermore blame you personally for all misgivings of the format! 😄

Jokes aside, I think I kind of see where this wants to come from, but I don't think any permission spec-side to mingle the multi- and single-scales' responsibilities will improve the format. This is the core value proposition, and it needs to be tight.

multiscale nodes are perfectly OK when they define only one scale. Adding another kind of node that may or may not know anything about itself, but if it does, it better be synchronised with someone else who-knows-where, provides no extra value. As you said: either

• the singlescale knows nothing -- and when outsourced from its parent multiscale, becomes nothing more than an indicator that "hey I'm part of an OME-Zarr of some kind" on the zarray itself, or
• the single scale needs to know its coordinate system and transforms -- then the multiscale attributes can disappear entirely.

I think the first option is more desirable because it's the model OME-Zarr has used for all previous versions, so it reduces learning overheads for implementers, and it keeps all OME-Zarr metadata in zarr groups rather than spreading it into zarr arrays (though that probably makes no substantial technical difference for pretty much any zarr back end). But as commented, it also makes the value of the singlescale as a node in the first place questionable, and it poses the question of whether the multiscale attributes should, as in past versions, structurally separate the transforms connecting its singlescales from the transforms connecting its additional coordinate systems and/or label multiscales (which would be more convenient for readers, but thanks to the unambiguous transform referencing isn't strictly necessary).

My suggestion would be in the direction of:

• multiscale nodes MUST contain only singlescale nodes and MUST contain at least one entry. The nodes MUST be ordered such that the shapes of the arrays at the singlescale nodes' paths never increase from start to end of the nodes.
• multiscale attributes MUST specify at least one coordinate system. They MUST also, per singlescale, specify exactly one transform of type identity, scale or sequence(scale[, translation]), whose input MUST reference the singlescale by path, and whose outputs MUST all reference the same ID of one of the multiscale's coordinate systems.
• singlescale nodes MUST NOT specify nodes, and MUST specify path, which MUST be of type zarr, and the object at the path MUST be a zarr array. The referenced array MAY declare itself as a singlescale node in its own zarr.json. singlescale attributes MAY be unset or empty, and MUST NOT specify coordinateSystems or coordinateTransformations MUST be empty
• (edit to add:) nodes that specify path MUST NOT specify attributes or nodes
• coordinateTransformations input and output MUST always reference a coordinate system ID, with one exception: when the input references a singlescale, it MUST be the singlescale node's ID

Here's the alternative form with multiscale-as-scene:

• singlescale nodes MUST have an id, their own nodes MUST be unset/empty/null, their attributes MUST specify exactly one coordinate system and MUST specify exactly one transform whose output MUST be the node's own coordinate system. The transform's input MAY be empty or null; if set it MUST be a Reference to the node's own ID
• (edit:) nodes that specify path MUST NOT specify attributes or nodes, except: singlescale nodes MUST specify both path and attributes
• multiscale nodes MUST contain only singlescale nodes and MUST contain at least one entry. The nodes MUST be ordered such that the shapes of the arrays at the singlescale nodes' paths never increase from start to end of the nodes. The nodes' coordinate systems MUST all have identical axes.
• multiscale attributes MAY be unset/empty/null. The attributes MUST NOT specify coordinateSystems or coordinateTransformations, but they MAY specify "scene" attributes. Any transforms therein that connect two singlescale nodes, directly or indirectly, MUST either be of type identity, or compose to the identity.
• coordinateTransformations input and output MUST always reference a coordinate system ID, with one exception: in singlescale attributes, the input MUST be either empty, or the singlescale node's ID

Disclaimer: I'm not sure the second variant actually works out like this. I'm more confident about the first variant. That's probably already one argument in favour of the first variant, it's just easier to reason about :)

Note how in the second variant, the multiscale itself can end up not knowing any axes if it outsources, and the only way to inline the pixel size scales and axes is to duplicate all the transforms and coordinate system (as you said). I think brushing over that tension with a hack doesn't solve it, like e.g.: "multiscale attributes must restate one coordinate system whose axes are identical with the singlescale nodes' axes, and must restate each transform from the singlescales' attributes; and singlescale nodes may omit their systems and transforms if they're inside a multiscale". One way or another, we end up with multiple possible manifestations of multiscales. The HCS spec has the same problem, but it's more tolerable there - it's just a bag of multiscales with little semantic connection between each other. I think doing the same to the multiscale could make OME-Zarr brittle to the point of potentially losing much or all of the interoperability it was supposed to deliver.

I guess it comes back to the question as written in the comment: I can't read the motivation for wanting outsourcable singlescale nodes out of the RFC. None of the use-cases in the RFC seem to need zarr arrays to become more annotated independently of a (multiscale) zarr group. The only related motivation I'm aware of is from the recent image.sc thread that makes it clear people want to remix existing arrays into new multiscales, but that doesn't need any metadata at all inside the array that's being referenced, no need for outsourcable singlescale nodes there...

(Wow I guess I had some time on this train ride. What an essay.)