[michael-johnston]

Update operator registration to read multiple resources of multiple types from the function.

---

_{Stack created with GitHub Stacks CLI • Give Feedback 💬}

[michael-johnston]

In this PR need to decide how to handle --with space and --use-latest space c.f. #1248

[AlessandroPomponio]

In this PR need to decide how to handle --with space and --use-latest space c.f. #1248

My suggestion would probably be to have a function that finds an input by name and returns the corresponding ADOResourceReference. We could then have:

• --use-latest <input-name>
• --with <input-name>=<id | resource-configuration>

Of course, --use-latest should be limited to one per kind, so there should be some validation to be had. As it stands, we're either asking people to provide the resource kind or deducing it, this suggestion would basically move it to only deducing it

[michael-johnston]

So to be clear given an operator with inputs foo and bar which are both kind DiscoverySpace you could do

--with foo=id  #bar takes the value it has in the YAML

but not --use-latest space as there are two spaces.

if you had input_space and timeseries_data of kind space and datacontainer though you could do

--use-latest space --use-latest dcr

?

[AlessandroPomponio]

I would completely deprecate the use of --use-latest <resource-type> and --with <resource-type>=<id | configuration>. We should have support for it in one version (2.4.0) but drop it completely in the next one (2.5.0).

I would require the name of the input to be specified, to keep things simple: --use-latest <input-name> and --with <input-name>=<id | configuration>

The only problem I see with this is how to handle sample stores and actuator configurations, since those aren't "inputs"

[michael-johnston]

So in the example above you'd have

--use-latest input_space --use-latest timeseries_data

seems a bit weird, and yes it won't work with samplestore or actuator configurations (same with with)

[AlessandroPomponio]

seems a bit weird

Why?

it won't work with samplestore or actuator configurations

What if we reserve those keywords?

[michael-johnston]

Why?

Because --use-latest [random-word] sounds strange, because you are not using the latest [random word]. You need to understand it means --use-latest-matching-kind-for-operator-input-name

[AlessandroPomponio]

Because --use-latest [random-word] sounds strange, because you are not using the latest [random word]. You need to understand it means --use-latest-matching-kind-for-operator-input-name

An alternative could be to have:

--use-latest <resource-type>=<input-name>
--with <input-name>=<id | resource-configuration>

In your example:

--use-latest space=input_space --use-latest dcr=timeseries_data
--with input_space=<id | resource> --with timeseries_data=<id | resource>

Here, however, you'd have to make sure the matches that of input_space (i.e., --use-latest operation=input_space should be rejected)

[michael-johnston]

Lets step back to why --with and --use-latest exist.

1. Certain resource depend on other resources - still true
   • discoveryspace -> needs a samplestore
   • explore operations -> needs a space and optionally and actuatorconfiguration
2. Where the dependency comes from is different
   • samplestore and actuatorconfiguration are known fields of the ado model
   • space is related to the input field of the "plugin" explore operator
3. We know explore operators take one space regardless of how it is named - this allowed us to treat "space" like a known dependency (still true)
4. --with and --use-latest provided us quick ways of setting these dependencies without editing YAML or knowing exact JSON paths for "set" or having to copy the exact resource id (as usually it was the last created).

Then because other operators could also take only one space, we could apply (3) to them also. Now we have the problem that this is not true anymore and operators can have multiple resource inputs of multiple types with multiple names. However 1-3 are still true.

Basically we have two types of "inputs" to resources; dependencies: The resources an ado resource depends on (all known a priori defined in model) and; payload: the dynamically changing inputs to an operator function.

Some choices that gives us:

A. We keep use-latest and --with for cases 1-3 which are still valid and don't support other operator types anymore. This keeps the reason these flags exist intact. i.e. use-latest and with are for simplifying creating discoveryspace and explore operations
B. We try to find a way to have the multi-input/multi-type operators work with --use-latest and --with. Problem is they were not designed for this case, leading to difficulties observed in previous comments.
- B1. We could treat --use-latest and --with differently
C. We could have a different flag for operator inputs e.g. --with-op-input
...

[michael-johnston]

@AlessandroPomponio Here's a suggestion

--with [kind][:name (optional) = ID
--use-latest  [kind][:name (optional)]

examples

--with space=ID                     # unchanged: deduce the input name from the kind
--with space:reference_space=ID     # explicit input name
--use-latest space                  # unchanged: deduce
--use-latest space:reference_space  # explicit input name
# an operator which takes two spaces. Provide one via with, and use the latest for the comparison_space. 
--with space:reference_space=ID --use-latest space:comparison_space
# an operator which takes a space and a datacontainer. 
--use-latest space --use-latest dcr #existing shorthand is fine
--with space=$ID --with dcr=$ID #same with `--with`, seems natural. 

• The flag still means "the latest space", and the input name only narrows where it goes
  - We already do a deductive process for --with and --use-latest. It's not new.
• Nothing has to be deprecated. Explore operators, samplestore and actuatorconfiguration all keep working as they do today.
• Validation is simple. The qualified input must exist, and its kind must match the prefix. If deducation cannot work a clear error is raised. Same with If an id is specified twice.
• It reads clearly to me.

This is similar to your = idea, but with the kind first in both flags, so --with and --use-latest have the same shape. In your version, --with uses input=id and --use-latest uses type=input, so = means different things in each flag.

The only thing is the : separator needs checking.

[michael-johnston]

Let's leave --with/--use-latest as it is in this branch. It will work in all cases as before but will fail for operations with multiple space inputs.