Added ZIP support spec#1717
Conversation
@check-spelling-bot ReportUnrecognized words, please review:
Previously acknowledged words that are now absentactivatable Globals mytoolTo accept these unrecognized words as correct (and remove the previously acknowledged and now absent words), run the following commands... in a clone of the git@github.com:midoriiro/winget-cli.git repository |
@check-spelling-bot ReportUnrecognized words, please review:
Previously acknowledged words that are now absentactivatable Globals mytoolTo accept these unrecognized words as correct (and remove the previously acknowledged and now absent words), run the following commands... in a clone of the git@github.com:midoriiro/winget-cli.git repository |
@check-spelling-bot ReportUnrecognized words, please review:
Previously acknowledged words that are now absentactivatable Globals mytoolTo accept these unrecognized words as correct (and remove the previously acknowledged and now absent words), run the following commands... in a clone of the git@github.com:midoriiro/winget-cli.git repository |
@check-spelling-bot ReportUnrecognized words, please review:
Previously acknowledged words that are now absentactivatable Globals mytoolTo accept these unrecognized words as correct (and remove the previously acknowledged and now absent words), run the following commands... in a clone of the git@github.com:midoriiro/winget-cli.git repository |
Exact, did not notice that. I just clone with git bash. I'll try to fix that. Edit: should be better now |
f013719 to
d05d7d9
Compare
This comment has been minimized.
This comment has been minimized.
| First thing first, the archive need to be extracted in a specific path (TEMP folder can be a good candidate). | ||
|
|
||
| Regarding case 1, installation flow check if TEMP folder contain a single binary and repack it | ||
| into an MSI package (including setting PATH env var). Then, MSI installation flow can take control. |
There was a problem hiding this comment.
Packaging a standalone / portable application is an interesting approach. I'm not sure we had considered this specific approach. We may prefer the concept of repackaging as an MSIX as that is the latest technology, but there may be different concerns to discuss here.
There was a problem hiding this comment.
I'm not sure that writing code to transform the archive into an installable type is better than simply writing the code to manage archives directly. All of the proposals require unpacking the archive, which is itself the primary step in management. Afterward, adding to the %PATH% is fairly straightforward and the issues are more about lifetime management. Injecting our own ARP entry is largely sufficient to achieve that.
There was a problem hiding this comment.
@denelon I'm not an expert concerning Windows packaging application. The main idea of this specification is to package the content of a zip file (except for case 2) in order to let winget managing the lifecycle of the new package.
I saw Windows SDK offers a set of CLI command to create MSIX package, but that mean installing Windows SDK.
@JohnMcPMS I don't know what is ARP, but if I understood your suggestion. How you do you handle uninstall/upgrade of the package ? That the goal of an ARP entry ?
There was a problem hiding this comment.
Add Remove Programs (ARP) is where many of the entries for programs installed on Windows 10 and Windows 11 get recorded. We use that metadata for the list command and as the source for values we used to correlate with manifests in configured sources. We don't necessarily need to create an "insaller" per se'. It's possible to just write entries to ARP (now part of Apps & Features). The Windows Package Manager could act as an "installer" proxy to create the entries and modify the path environment variable. We're looking at pros and cons to several approaches.
There was a problem hiding this comment.
I see, it's make sense. I looked if an API exist about managing an ARP entry, but I did found nothing. If winget-cli already manage that kind of tasks, @JohnMcPMS made a good point.
I don't know what is the state of art about ARP in Windows 10/11 now. But, ten years ago I wrote an application that find all installed applications on the system through the registry, the main goal was to uninstall silently a batch of applications. That was tricky to handle all the cases.
If an API already exist, the repacking solution would not be the best approach(in my opinion) as you explained both of you.
I'll try to find some documentation on the internet and this repository to study how to manage an ARP entry.
There was a problem hiding this comment.
I don't think that there are APIs; we certainly do not use them and I looked for them when we implemented reading. It is basically a registry data contract.
There was a problem hiding this comment.
Yes, I saw that yesterday. I updated the spec. There some points to discuss about it but, especially where to install the zip content. The ARP section seem good for you ?
There was a problem hiding this comment.
Perhaps we treat ZIP installs the same way visual studio code is installed (i.e. user installer) within the AppData folder.
There was a problem hiding this comment.
That I suggested in the last revision of the spec. For user scope, application will be installed in AppData/Local/myapplication. But maybe there is a better location for user scope ?
In any ways, scope and location can be overrides by users when calling winget install. Just found that, that should be take in consideration on the specification.
|
|
||
| ### Security | ||
|
|
||
| No direct impact on security. |
There was a problem hiding this comment.
There would most certainly be security implications. We may even need a SHA256 hash for each of the files to be packaged. We don't want to be in a state where the files are downloaded and modified prior to the packaging steps. This will also have a substantial impact on the validation flow so we can verify the hashes are valid at validation time, and correctly included in the manifest for any changes to the manifest.
There was a problem hiding this comment.
That could be tricky to add all checksum for all files in the manifest. If the ZIP archive is validated can we just lock the extracted files before packaging to prevent any modifications by other process ?
|
|
||
| ### Compatibility | ||
|
|
||
| No code breaking, reuse existing code and add code to support zip installation. |
There was a problem hiding this comment.
This would not be a backward compatible behavior. Only clients with the support for this feature would be able to install packages of this type. We would need to add something like "zip" to the enumeration of installer types. The initial implementation would also be an experimental feature until we have covered enough of the different use cases to confirm viability of the solution.
There was a problem hiding this comment.
Yes, you are right. For the moment this draft is only intended to schema 1.1.0. Like i suggested, we maybe need to update the schema (1.1.1?) to handle case 3 and other new cases.
|
|
||
| No code breaking, reuse existing code and add code to support zip installation. | ||
|
|
||
| ### Performance, Power, and Efficiency |
There was a problem hiding this comment.
Building the installer is likely to take additional time and resources on the client. There will be performance considerations here with the client in the packaging phase.
|
|
||
| ### Performance, Power, and Efficiency | ||
|
|
||
| ## Potential Issues |
There was a problem hiding this comment.
MSIX packages are typically signed. There would likely be security issues related to signing. If the MSI route or some other mechanism is used for the package, we would also need to look at those considerations. Even if we just move the standalone / portable application (or set of files) to a directory path, any changes to those files could have unintended consequences. Some packages also generate additional files in their directories so those may need to be considered during an upgrade or uninstall flow.
There was a problem hiding this comment.
if we just move the standalone / portable application (or set of files) to a directory path, any changes to those files could have unintended consequences.
which is true of any package winget currently installs, as with admin rights (or for user scoped packages, normal rights) I can go mess around with anything the installer created. I think that might be a separate issue.
There was a problem hiding this comment.
Like someone said on issue #140, if a file is modified is not the packager responsibility but it's an user fault. If files are created by a process delivered by the package, these files should be referenced somewhere to prevent overwriting them, or rather we can specify to overwrite theses files during an upgrade. Typically for config files, user need to edit them, that could be a problem if on each upgrade these files are overwritten.
Concerning signing a MSIX package, A self-signed package could be acceptable ?
|
Thanks for taking the time to write this draft PR! Please be patient as we're dealing with limited resources during this time of year. We also need time for additional internal design review, and have a security review performed to make sure we aren't facing any other additional security concerns. There may also be licensing or copyright concerns to consider if the publisher or the artifacts aren't governed by an open source license. |
| First thing first, the archive need to be extracted in a specific path (TEMP folder can be a good candidate). | ||
|
|
||
| Regarding case 1, installation flow check if TEMP folder contain a single binary and repack it | ||
| into an MSI package (including setting PATH env var). Then, MSI installation flow can take control. |
There was a problem hiding this comment.
I'm not sure that writing code to transform the archive into an installable type is better than simply writing the code to manage archives directly. All of the proposals require unpacking the archive, which is itself the primary step in management. Afterward, adding to the %PATH% is fairly straightforward and the issues are more about lifetime management. Injecting our own ARP entry is largely sufficient to achieve that.
| ManifestVersion: 1.0.0 | ||
| ``` | ||
|
|
||
| This is not compliant with the actual schema v1.1.0. We need to specify which binary to add into |
There was a problem hiding this comment.
Currently you would get an ERROR_NOT_SUPPORTED result. If you think that isn't a good experience, we should come up with a better one and implement now to increase the chances of the downlevel client having it.
| - Architecture: x64 | ||
| InstallerUrl: https://editor.com/app/1.0.zip | ||
| InstallerSha256: ... | ||
| BinaryPath: bin/binary.exe |
There was a problem hiding this comment.
If this is intended to be the %PATH% update location, it should be to a directory and not a file. The default would be either be no %PATH% update or the root.
There was a problem hiding this comment.
Yes, I noticed my mistake before your write your comment ^^". You are right, BinaryPath should be a relative path from the root archive.
|
|
||
| Take a look into the following manifest examples (v1.1.0) | ||
|
|
||
| ### Case 1|2 YAML manifest |
There was a problem hiding this comment.
I would argue for case 2 to be the one that includes a relative path to the installer, in the event that there are support files included within the archive.
There was a problem hiding this comment.
Case 2 expect to have an installer at the root archive. But if the installer is located to 'archive.zip/somefolder/installer.exe' that should be a new case in my opinion. To avoid having more case I suggest to add a new attribute like BinaryPath.
@check-spelling-bot ReportUnrecognized words, please review:
Previously acknowledged words that are now absentactivatable Globals mytoolTo accept these unrecognized words as correct (and remove the previously acknowledged and now absent words), run the following commands... in a clone of the git@github.com:midoriiro/winget-cli.git repository |
|
@denelon I know this feature is not planned to be released shortly. However, I'll take on consideration your feedback (@JohnMcPMS included) and push a new revision soon, so we can discuss it when you will be available ;-) |
@check-spelling-bot ReportUnrecognized words, please review:
Previously acknowledged words that are now absentactivatable Globals mytoolTo accept these unrecognized words as correct (and remove the previously acknowledged and now absent words), run the following commands... in a clone of the git@github.com:midoriiro/winget-cli.git repository |
@check-spelling-bot ReportUnrecognized words, please review:
Previously acknowledged words that are now absentactivatable Globals mytoolTo accept these unrecognized words as correct (and remove the previously acknowledged and now absent words), run the following commands... in a clone of the git@github.com:midoriiro/winget-cli.git repository |
@check-spelling-bot ReportUnrecognized words, please review:
Previously acknowledged words that are now absentactivatable Globals mytoolTo accept these unrecognized words as correct (and remove the previously acknowledged and now absent words), run the following commands... in a clone of the git@github.com:midoriiro/winget-cli.git repository |
| - Architecture: x64 | ||
| InstallerUrl: https://editor.com/app/1.0.zip | ||
| InstallerSha256: ... | ||
| BinaryPath: bin (e.g. bin folder where vagrant.exe is located in my example) |
There was a problem hiding this comment.
Could this be simplified to "InstallerPath" as well, and have the cli determine if it is an installer or binary?
There was a problem hiding this comment.
Case 2 cover installer (.exe, .msi, etc). You're pointing case 3, which is for complex hierarchy that does not include installer, only binary with complex folders structure. The base idea is to implement all the cases. Nonetheless, "BInaryPath" maybe not a good name, any idea ?
There was a problem hiding this comment.
I’m saying that for case 2, you are suggesting that a key be added to define the path to the exe/msi/etc.
Why can’t that same key be used for case 3?
There was a problem hiding this comment.
I got confused you was quoting case 3. On case 3, BinaryPath key is for indicate to register a path after the zip extraction into user or system environment variables (need to be defined how to).
Correct me if I'm wrong, but from my point of view regarding case 2 (excluding my example manifest 2.1). Case 2 assuming that the zip archive only contains one file, the installer. Case 2 also assuming that the installer will set the proper environment variables if needed (specifically $PATH) or not. that depend of the nature of the application and how editors package their applications. I don't know, how do you manage that kind of thing, winget-cli is agnostic regarding this aspect ?
To answer your question, case 3 does not handle installer file. That the job of case 2 and 2.1. Case 2 does not need to define InstallerPath key because, the archive only contains the installer at the root archive. On the other hand, case 2.1 zip archive maybe not contains only one file and maybe not located at root, so InstallerPath key is needed to indicate where to find the installer after the extraction and execute it.
To summarize, BinaryPath and InstallerPath are optional keys that depend on the context and they should be set properly by the package maintainer. To avoid messing up with these settings, updating the documentation and the helper script YAMLCreate.ps1 should be needed, in my opinion. For all cases, if the manifest is miss-configured the procedure should fail in an organic way.
I don't know if I'm clear, my english is not perfect. Maybe I should clarify these aspects on the specification.
| - Architecture: x64 | ||
| InstallerUrl: https://editor.com/app/1.0.zip | ||
| InstallerSha256: ... | ||
| InstallerPath: somefolder/installer.exe |
There was a problem hiding this comment.
I think we may also need a key defining the secondary installer type.
It is likely that some of the exe’s shipped inside zip packages are actually nullsoft or inno, which should be preferred to exe since the CLI already handles their silent switches without having to define them
There was a problem hiding this comment.
A key definition already exist on schema 1.1.0 (line 459)
Maybe we can use it to tell which installation flow to choose?
There was a problem hiding this comment.
That key, by definition, would be "zip" since the primary installer is inside of a .zip file. The issue I'm trying to get ahead of is that the zip file could contain different types of exes which have their own installation flow
There was a problem hiding this comment.
The key I'm pointing could be resolve that issue with something like this:
PackageIdentifier: Editor.App
PackageVersion: 1.0
MinimumOSVersion: 10.0.0.0
InstallerType: zip
Scope: ...
InstallModes: ...
InstallerSwitches:
Silent: ...
SilentWithProgress: ...
UpgradeBehavior: install
Installers:
- Architecture: x64
InstallerUrl: https://editor.com/app/1.0.zip
InstallerSha256: ...
InstallerPath: somefolder/installer.exe
InstallerType: inno # or something else
ManifestType: installer
ManifestVersion: 1.0.0
There was a problem hiding this comment.
I don't believe that is valid. Having the key at the manifest level is not compatible with the key at the installer level. One of them will override the other.
Then, what if a publisher distributes some apps via msi and others via zip => nullsoft ? Its possible that not all of the installers inside of a manifest will be zipped, and so we would need to use the installer level key to specify which ones are zip files and which ones are msi
There was a problem hiding this comment.
I’m not saying multiple installers may be in a .Zip archive. I’m saying a manifest may have different types of installers, and a zip archive may be one of them.
For example - the 7-zip manifest has installers for both exe and msi. Both are in the same manifest.
Some publishers pubish exe’s, msi’s, and zip files, which should all be able to be in a single manifest
There was a problem hiding this comment.
Ok I understand now. And I agree with you about defining another key to specify the installer type. Maybe InstallerSubType ?
I need to clarify some points on the specification about previous conversations. I'll commit soon.
There was a problem hiding this comment.
#1242 is how we're planning to capture some of the "nested installer type" issues. For example, PowerToys is distributed as an .exe, but it installs as an .msi. The installer is a wrapper to help with dependencies.
There was a problem hiding this comment.
Perfect, I think that as long as that issue is referenced in the spec, then there won't be an issue here
There was a problem hiding this comment.
That should close this issue. Thank for your feedback.
@check-spelling-bot ReportUnrecognized words, please review:
Previously acknowledged words that are now absentactivatable Globals mytoolTo accept these unrecognized words as correct (and remove the previously acknowledged and now absent words), run the following commands... in a clone of the git@github.com:midoriiro/winget-cli.git repository |
|
What's the state about my last commits? Someone has time to review them? I know you're busy, I'm just checking. Thanks |
|
We're getting ramped up on how we're going to tackle .zip support. We will take as much of what has been drafted as possible. |
|
Hey @midoriiro, The team has begun working on a design for adding .ZIP installer support. As this spec is a great starting point and a lot of important discussions have taken place, we plan to discuss and incorporate key components that you have brought to our attention into our design review. We will continue to update the community as we make progress. Great work :) |
|
We're getting closer with support for portable applications: Once that is "code complete", we will turn our attention back to .zip. |
|
Superseded by #2270 |
411a10915 mint 2.10.19 096a9a1bb Merge pull request microsoft#1782 from microsoft/fix_uri_parsing 006271f6a make Uri.is_host_loopback() only return true for localhost and 127.0.0.1 exactly 9c654889e Remove email list from the readme 31e7feacc Merge pull request microsoft#1718 from dashanji/Fix-typo e1b6a8e61 Merge pull request microsoft#1711 from Fighter19/pr-fix-safeint3 c5dcbb5bb Merge pull request microsoft#1717 from microsoft/users/GitHubPolicyService/f2ee14d6-8d6b-4313-b754-3880e015f7c3 98ee36dac Fix typo a57f45918 Microsoft mandatory file 3308d9728 Fix likely typo in SafeInt3.hpp, that results in error with clang 15 07cf58910 Merge pull request microsoft#1429 from NN---/fix/value_int_ctor 3eac925ad Update Release/include/cpprest/json.h 06363bc78 Update Release/include/cpprest/json.h bfe348779 Merge pull request #1577 from JvdGlind/hidden_visibility_support_macos 0ddc61829 Clarify cpprestsdk level of support 804448058 export http_exception for non Windows builds d9d7f5ed4 Merge pull request microsoft#1496 from icherniukh/oauth2_client_credentials 8ae5da616 Update oauth2.h 708a5df2b Add support for oauth2 using only client credentials 5408f1dc9 Add constructor from all integer types. git-subtree-dir: src/cpprestsdk/cpprestsdk git-subtree-split: 411a109150b270f23c8c97fa4ec9a0a4a98cdecf
411a10915 mint 2.10.19 096a9a1bb Merge pull request microsoft#1782 from microsoft/fix_uri_parsing 006271f6a make Uri.is_host_loopback() only return true for localhost and 127.0.0.1 exactly 9c654889e Remove email list from the readme 31e7feacc Merge pull request microsoft#1718 from dashanji/Fix-typo e1b6a8e61 Merge pull request microsoft#1711 from Fighter19/pr-fix-safeint3 c5dcbb5bb Merge pull request microsoft#1717 from microsoft/users/GitHubPolicyService/f2ee14d6-8d6b-4313-b754-3880e015f7c3 98ee36dac Fix typo a57f45918 Microsoft mandatory file 3308d9728 Fix likely typo in SafeInt3.hpp, that results in error with clang 15 07cf58910 Merge pull request microsoft#1429 from NN---/fix/value_int_ctor 3eac925ad Update Release/include/cpprest/json.h 06363bc78 Update Release/include/cpprest/json.h bfe348779 Merge pull request #1577 from JvdGlind/hidden_visibility_support_macos 0ddc61829 Clarify cpprestsdk level of support 804448058 export http_exception for non Windows builds d9d7f5ed4 Merge pull request microsoft#1496 from icherniukh/oauth2_client_credentials 8ae5da616 Update oauth2.h 708a5df2b Add support for oauth2 using only client credentials 5408f1dc9 Add constructor from all integer types. git-subtree-dir: src/cpprestsdk/cpprestsdk git-subtree-split: 411a109150b270f23c8c97fa4ec9a0a4a98cdecf
Added ZIP support spec for community review as draft
Microsoft Reviewers: Open in CodeFlow