Redesign update-dependencies tool to better support internal builds #6305

Background / Problem

There are a few things contributing to this design proposal:

• The update-dependencies tool needs support for internal builds of products besides .NET (YARP, Monitor, etc.).
• It relies on lots of outside PowerShell scripts to determine the correct version of things. This setup is fragile and messy.
• It can only update one .NET version at a time. We want the tool to update multiple versions at once for the purpose of regularly updating and staging multiple internal .NET builds.

I'd also like the tool to be flexible/configurable enough that adding support for new products or tools doesn't require changes to code (or requires very few changes). Adding new automatic product/version updates should be simple and driven by configuration, not hard-coded.

Related: #6252, https://github.com/dotnet/dotnet-docker-internal/issues/6782

Design proposal

The tool should be able to update any number of .NET image components at once.

The idea is for this to replace or drastically simplify the following scripts:

• eng/Set-DotNetVersions.ps1
• eng/Get-DropVersions.ps1
• eng/Get-AspireDropVersions.ps1
• eng/Get-MonitorDropVersions.ps1

Updates can be:

• Automatic (latest version) via build channel (10.0.1xx-preview2/daily)
• Precise versions (10.0.0-preview.1.25120.3, 8.0.201)
• Azure DevOps pipeline run IDs (1234567)
• Internal or public

Desired UX

Update .NET using SDK build channel:

• --sdkChannel "10.0.100-preview2/daily"
• --sdkChannel "10.0.1xx/daily"

Update to a specific (publicly released) .NET version:

• --version "sdk=8.0.30X,aspnet=8.0.X,runtime=8.0.X"

Update tools:

• Update tool to latest for all product versions: --tool "chisel"
• Update tool to latest for specific product versions: --tool "chisel=1.0.2"
• Specific tool version for specific product version: --tool "productVersion=8.0,chisel=1.0.2"

Update from a build ID:

• --buildId "1234567"

Update a specific product from a build ID:

• --buildId "product=monitor,build=1234567"
• Assume the product is the .NET SDK if none is specified.

The Dockerfile versions to update will be determined by the first two version parts of the product build version.
If that's not sufficient (such as when updating between minor versions) it can be overridden:

• --buildId "product=monitor,dockerfileVersion=9.0,build=1234567"

Update from multiple build IDs:

• --buildId "1234567,2222222,4567890"

Set up Dockerfiles for internal builds:

• --internalBuildId "1234567"
• --internalBuildId "1234567,2222222,4567890"

Config

Given an AzDO pipeline run ID, the tool should be able to extract product versions and download paths from the pipeline.

Most build/release pipelines publish an artifact describing what they've published.
We can use a config file to tell the tool how to extract build information that file.
Here's an example of what that might look like.

{
  "pipelines": [
    {
      "id": "13XX",
      "config": {
        "kind": "json",
        "path": "drop/config.json"
      },
      "products": {
        "sdk": {
          "versionSelector": "Sdk_Builds",
          "multipleVersionRule": "highest",
          "downloadPath": "shipping/assets/Sdk"
        },
        "aspnet": {
          "versionSelector": "Asp",
          "downloadPath": "shipping/assets/aspnetcore/Runtime"
        },
        "runtime": {
          "versionSelector": "Runtime",
          "downloadPath": "shipping/assets/Runtime"
        }
      }
    },
    {
      "id": "8XX",
      "config": {
        "kind": "xml",
        "path": "BlobArtifacts/MergedManifest.xml",
      },
      "products": {
        "yarp": {
          "versionSelector": "//Build/Package[@Id='yarp']/@Version",
          "downloadPath": "BlobArtifacts/"
        }
      }
    },
    {
      "id": "13XX",
      "config": {
        "kind": "xml",
        "path": "BlobArtifacts/MergedManifest.xml",
      },
      "products": {
        "aspire-dashboard": {
          "versionSelector": "//Build/Package[@Id='Aspire.Dashboard.Sdk.linux-x64']/@Version",
          "downloadPath": "BlobArtifacts/"
        }
      }
    }
  ]
}