Skip to content

Enable JSON schema validation for import#779

Merged
florelis merged 6 commits into
microsoft:masterfrom
florelis:PackagesJsonValidation
Mar 5, 2021
Merged

Enable JSON schema validation for import#779
florelis merged 6 commits into
microsoft:masterfrom
florelis:PackagesJsonValidation

Conversation

@florelis

@florelis florelis commented Mar 4, 2021

Copy link
Copy Markdown
Member

Enabling validation of JSON files for import according to the schema.

  • Extracted common logic from existing manifest JSON validation.
  • Added new shared items project for the schema resource.
  • Added validation when reading the import file. The caller already handled the error case and needed no changes.
  • Enabled existing tests for this.
Microsoft Reviewers: Open in CodeFlow

@florelis florelis requested a review from a team as a code owner March 4, 2021 22:28
@github-actions

github-actions Bot commented Mar 4, 2021

Copy link
Copy Markdown

Misspellings found, please review:

  • PACKAGESSCHEMA
To accept these changes, run the following commands from this repository on this branch
pushd $(git rev-parse --show-toplevel)
perl -e '
my @expect_files=qw('".github/actions/spelling/expect.txt"');
@ARGV=@expect_files;
my @stale=qw('"validator valijson valueiterator "');
my $re=join "|", @stale;
my $suffix=".".time();
my $previous="";
sub maybe_unlink { unlink($_[0]) if $_[0]; }
while (<>) {
  if ($ARGV ne $old_argv) { maybe_unlink($previous); $previous="$ARGV$suffix"; rename($ARGV, $previous); open(ARGV_OUT, ">$ARGV"); select(ARGV_OUT); $old_argv = $ARGV; }
  next if /^(?:$re)(?:(?:\r|\n)*$| .*)/; print;
}; maybe_unlink($previous);'
perl -e '
my $new_expect_file=".github/actions/spelling/expect.txt";
use File::Path qw(make_path);
make_path ".github/actions/spelling";
open FILE, q{<}, $new_expect_file; chomp(my @words = <FILE>); close FILE;
my @add=qw('"PACKAGESSCHEMA "');
my %items; @items{@words} = @words x (1); @items{@add} = @add x (1);
@words = sort {lc($a) cmp lc($b)} keys %items;
open FILE, q{>}, $new_expect_file; for my $word (@words) { print FILE "$word\n" if $word =~ /\w/; };
close FILE;'
popd

Comment thread src/Schemas/Schemas.vcxitems Outdated
@florelis florelis force-pushed the PackagesJsonValidation branch from 7e88d6e to eed5fa4 Compare March 5, 2021 00:56
Comment thread src/AppInstallerCLI.sln Outdated
Comment thread src/AppInstallerCLICore/PackageCollection.cpp Outdated
{
// TODO: Embed schema in binaries & validate file. This will return nullopt on failure.
// Validate the JSON against the schema.
Json::Value schemaJson = JsonSchema::LoadResourceAsSchemaDoc(MAKEINTRESOURCE(IDX_PACKAGES_SCHEMA_V1), MAKEINTRESOURCE(PACKAGESSCHEMA_RESOURCE_TYPE));

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LoadResourceAsSchemaDoc [](start = 49, length = 23)

How will we determine which schema to load in the future when there are more than one version? That can probably be a problem for future developer, but let's consider if we are just putting off the work or if we are making that work harder.

For instance, will a file with no "schema" json item validate as good? Do we want to support that and thus default to V1 schema if no value present? Or would we prefer to require the "schema" to be present and that it match the value that we expect, regardless of version.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the future we could load the schema depending on the $schema property, and have some fall back if it is missing or not recognized (e.g. use latest schema).

This does not validate that the file has a $schema property or that it has a specific value. We may be able to require it in the schema if we wanted to.

If we create newer versions this function will accept them as long as they are backwards compatible. All the nodes accept additional unrecognized properties to allow room for extending without causing validation failures.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should require the $schema property and that it contains the expected value, even if we don't actually use it to select the schema to use yet. This seems like something that should be done by the code rather than in the schema file itself.

Comment thread src/WinGetSchemas/resource.h Outdated

@JohnMcPMS JohnMcPMS left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@florelis florelis merged commit 9e104eb into microsoft:master Mar 5, 2021
@florelis florelis deleted the PackagesJsonValidation branch April 2, 2021 23:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants