Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions NuGet.Config
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,7 @@
<packageSources>
<!--To inherit the global NuGet package sources remove the <clear/> line below -->
<clear />
<add key="myget-legacy" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/myget-legacy/nuget/v3/index.json" />
<add key="dotnet5" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5/nuget/v3/index.json" />
<add key="dotnet3.1" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet3.1/nuget/v3/index.json" />
<add key="dotnet7" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet7/nuget/v3/index.json" />
<add key="dotnet-public" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-public/nuget/v3/index.json" />
</packageSources>
<packageRestore>
Expand Down
46 changes: 23 additions & 23 deletions doc/diffs.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,8 @@ To build jitutils not using the bootstrap script:
* Run `build.{cmd,sh}`.

By default the script just builds the tools and does not publish them in a separate directory.
To publish the utilities add the '-p' flag which publishes each utility to the ./bin directory
in the root of the repo. Additionally, to download the default set of framework assemblies
To publish the utilities add the '-p' flag which publishes each utility to the ./bin directory
in the root of the repo. Additionally, to download the default set of framework assemblies
that can be used for generating asm diffs, add '-f'.

```
Expand All @@ -66,23 +66,23 @@ build.sh [-b <BUILD TYPE>] [-f] [-h] [-p] [-t <TARGET>]
By default, assembly code output (aka, "dasm") is generated by running crossgen
with a specified JIT to compile a specified set of assemblies, by setting the
following JIT environment variables to generate the output:
* `COMPlus_NgenDisasm`
* `COMPlus_NgenUnwindDump`
* `COMPlus_NgenEHDump`
* `COMPlus_JitDiffableDasm`
* optionally, `COMPlus_NgenGCDump`
* `DOTNET_NgenDisasm`
* `DOTNET_NgenUnwindDump`
* `DOTNET_NgenEHDump`
* `DOTNET_JitDiffableDasm`
* optionally, `DOTNET_NgenGCDump`

Generating "diffs" involves generating assembly code output for both a baseline
and a "diff" JIT, and comparing the results.

Passing the `--pmi` option to `jit-diffs` will instead use reflection to jit each
method in the assembly, setting these options:
* `COMPlus_JitDisasm`
* `COMPlus_JitUnwindDump`
* `COMPlus_JitEHDump`
* `COMPlus_JitDiffableDasm`
* `COMPlus_JitDisasmAssemblies`
* optionally, `COMPlus_JitGCDump`
* `DOTNET_JitDisasm`
* `DOTNET_JitUnwindDump`
* `DOTNET_JitEHDump`
* `DOTNET_JitDiffableDasm`
* `DOTNET_JitDisasmAssemblies`
* optionally, `DOTNET_JitGCDump`

## What can jit-diff produce asm diffs for?

Expand Down Expand Up @@ -168,7 +168,7 @@ The "jit-diff diff" command has this help message:
[-t <arg>] [-c] [-f] [--benchmarks] [--tests] [--gcinfo] [-v] [--core_root <arg>]
[--test_root <arg>] [--base_root <arg>] [--diff_root <arg>] [--arch <arg>]
[--build <arg>] [--altjit <arg>]

-b, --base [arg] The base compiler directory or tag. Will use crossgen or clrjit from this
directory.
-d, --diff [arg] The diff compiler directory or tag. Will use crossgen or clrjit from this
Expand All @@ -195,35 +195,35 @@ The "jit-diff diff" command has this help message:
--altjit <arg> If set, the name of the altjit to use (e.g., clrjit_win_arm64_x64.dll).
--pmi Generate diffs via jitting instead of running crossgen
--assembly <arg> Look at diffs for methods in the specified assembly

Examples:

jit-diff diff --output c:\diffs --corelib --core_root c:\coreclr\bin\tests\windows.x64.Release\Tests\Core_Root --base c:\coreclr_base\bin\Product
\windows.x64.Checked --diff c:\coreclr\bin\Product\windows.x86.Checked
Generate diffs of System.Private.CoreLib.dll by specifying baseline and
diff compiler directories explicitly.

jit-diff diff --output c:\diffs --base c:\coreclr_base\bin\Product\windows.x64.Checked --diff
If run within the c:\coreclr git clone of dotnet/coreclr, does the same
as the prevous example, using defaults.

jit-diff diff --output c:\diffs --base --base_root c:\coreclr_base --diff
Does the same as the prevous example, using -base_root to find the base
directory (if run from c:\coreclr tree).

jit-diff diff --base --diff
Does the same as the prevous example (if run from c:\coreclr tree), but uses
default c:\coreclr\bin\diffs output directory, and `base_root` must be specified
in the config.json file in the directory pointed to by the JIT_UTILS_ROOT
environment variable.

jit-diff diff --diff
Only generates asm using the diff JIT -- does not generate asm from a baseline compiler --
using all computed defaults.

jit-diff diff --diff --arch x86
Generate diffs, but for x86, even if there is an x64 compiler available.

jit-diff diff --diff --build Debug
Generate diffs, but using a Debug build, even if there is a Checked build available.
```
Expand Down Expand Up @@ -380,7 +380,7 @@ Or, disassemble the jitted code for all the methods in `mytest.exe`:
```

Note this latter run should produce similar disassembly as running `mytest.exe` via
corerun (with appropriate COMPlus flags set) for the methods that are executed
corerun (with appropriate DOTNET_ flags set) for the methods that are executed
during the run. But `jit-diff diff -pmi` will attempt to show code generated for
all methods, executed or not. And it also works on libraries which are not
directly executable on their own. So PMI offers a potentially faster and more
Expand Down
6 changes: 1 addition & 5 deletions src/AnalyzeAsm/AnalyzeAsm.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,7 @@

<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp3.1</TargetFramework>
<TargetFramework>net6.0</TargetFramework>
</PropertyGroup>

<ItemGroup>
<PackageReference Include="Newtonsoft.Json" Version="12.0.3" />
</ItemGroup>

</Project>
6 changes: 3 additions & 3 deletions src/AnalyzeAsm/OccurenceInfo.cs
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,12 @@
// The .NET Foundation licenses this file to you under the MIT license.
// See the LICENSE file in the project root for more information.

using Newtonsoft.Json;
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Reflection;
using System.Text.Json;
using MethodDB = System.Collections.Generic.Dictionary<string, System.Collections.Generic.Dictionary<string, System.Collections.Generic.List<AnalyzeAsm.PositionInfo>>>;

namespace AnalyzeAsm
Expand Down Expand Up @@ -135,7 +135,7 @@ private static MethodIndex CreateIndex(string folderPath)
index.AddFile(dasmFile);
}

string contents = JsonConvert.SerializeObject(index);
string contents = JsonSerializer.Serialize(index);
File.WriteAllText(Path.Combine(folderPath, indexFileName), contents);

stopWatch.Stop();
Expand All @@ -158,7 +158,7 @@ private static bool TryGetIndex(string folderPath, out MethodIndex index, bool d
return false;
}

index = JsonConvert.DeserializeObject<MethodIndex>(File.ReadAllText(indexFilePath));
index = JsonSerializer.Deserialize<MethodIndex>(File.ReadAllText(indexFilePath));
var expectedExeTimeStamp = index.LastModifiedTimeOfExe;
var actualExeTimeStamp = File.GetLastWriteTimeUtc(Assembly.GetExecutingAssembly().Location);

Expand Down
8 changes: 4 additions & 4 deletions src/AnalyzeAsm/README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
Notes:

* The code in `Program.cs` has **lot of redundant code**. Most of the methods are copied from previous methods with little tweak.
* The code in `Program.cs` has **lot of redundant code**. Most of the methods are copied from previous methods with little tweak.
* All the methods rely on `ngen_arm64.txt` / `ngen_amd64.txt` file that are produced by doing the following:
* `set COMPlus_NGenDisasm=1`
* Running `build-test.cmd crossgen > ngen_arm64.txt`
* The path locations of these files are hardcoded too.
* `set DOTNET_NGenDisasm=1`
* Running `build-test.cmd crossgen > ngen_arm64.txt`
* The path locations of these files are hardcoded too.


`FindLdrGroups_1()` finds patterns:
Expand Down
18 changes: 9 additions & 9 deletions src/cijobs/cijobs.cs
Original file line number Diff line number Diff line change
Expand Up @@ -23,17 +23,17 @@


using System;
using System.IO;
using System.Collections.Generic;
using System.CommandLine;
using System.IO;
using System.IO.Compression;
using System.Linq;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Text.Json;
using System.Text.RegularExpressions;
using System.Threading;
using System.Threading.Tasks;
using System.CommandLine;
using System.Linq;
using System.Text.RegularExpressions;
using System.IO.Compression;
using Newtonsoft.Json;

namespace ManagedCodeGen
{
Expand Down Expand Up @@ -386,7 +386,7 @@ string productString
if (response.IsSuccessStatusCode)
{
var json = await response.Content.ReadAsStringAsync();
var productJobs = JsonConvert.DeserializeObject<ProductJobs>(json);
var productJobs = JsonSerializer.Deserialize<ProductJobs>(json);
return productJobs.jobs;
}
else
Expand Down Expand Up @@ -414,7 +414,7 @@ var messageString
if (response.IsSuccessStatusCode)
{
var json = await response.Content.ReadAsStringAsync();
var jobBuilds = JsonConvert.DeserializeObject<JobBuilds>(json);
var jobBuilds = JsonSerializer.Deserialize<JobBuilds>(json);

if (lastSuccessfulBuild)
{
Expand Down Expand Up @@ -492,7 +492,7 @@ public async Task<BuildInfo> GetJobBuildInfo(string repoName, string branchName,
if (response.IsSuccessStatusCode)
{
var buildInfoJson = await response.Content.ReadAsStringAsync();
var info = JsonConvert.DeserializeObject<BuildInfo>(buildInfoJson);
var info = JsonSerializer.Deserialize<BuildInfo>(buildInfoJson);
return info;
}
else
Expand Down
21 changes: 12 additions & 9 deletions src/jit-analyze/jit-analyze.cs
Original file line number Diff line number Diff line change
Expand Up @@ -3,17 +3,20 @@
// See the LICENSE file in the project root for more information.

using System;
using System.Diagnostics;
using System.Collections.Generic;
using System.CommandLine;
using System.Diagnostics;
using System.Globalization;
using System.IO;
using System.Collections.Generic;
using System.Linq;
using System.Text.RegularExpressions;
using Newtonsoft.Json;
using System.Text;
using System.Runtime.CompilerServices;
using System.Globalization;
using System.Reflection;
using System.Text;
using System.Text.Json;
using System.Text.Json.Serialization;
using System.Text.RegularExpressions;
using System.Threading;
using System.Threading.Tasks;
using System.Xml;

namespace ManagedCodeGen
{
Expand Down Expand Up @@ -397,7 +400,7 @@ static MetricCollection()
}
}

[JsonProperty()]
[JsonInclude]
private Metric[] metrics;

public MetricCollection()
Expand Down Expand Up @@ -1185,7 +1188,7 @@ public static StringBuilder GenerateJson(IEnumerable<FileDelta> compareList)
{
StringBuilder fileContents = new StringBuilder();

fileContents.AppendLine(JsonConvert.SerializeObject(compareList.Where(file => !file.deltaMetrics.IsZero()), Formatting.Indented));
fileContents.AppendLine(JsonSerializer.Serialize(compareList.Where(file => !file.deltaMetrics.IsZero()), new JsonSerializerOptions { WriteIndented = true }));

return fileContents;
}
Expand Down
48 changes: 24 additions & 24 deletions src/jit-dasm-pmi/jit-dasm-pmi.cs
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@

namespace ManagedCodeGen
{
// Define options to be parsed
// Define options to be parsed
public class Config
{
private ArgumentSyntax _syntaxResult;
Expand Down Expand Up @@ -481,56 +481,56 @@ void AppendEnvironmentVariableToPmiEnv(string varName, string varValue)
}
}

// Pick up ambient COMPlus settings.
// Pick up ambient DOTNET settings.
foreach (string envVar in Environment.GetEnvironmentVariables().Keys)
{
if (envVar.IndexOf("COMPlus_") == 0)
if (envVar.IndexOf("DOTNET_") == 0)
{
string value = Environment.GetEnvironmentVariable(envVar);
AppendEnvironmentVariableToPmiEnv(envVar, value);
}
}

// Set up environment do PMI based disasm.
AppendEnvironmentVariableToPmiEnv("COMPlus_JitDisasm", "*");
AppendEnvironmentVariableToPmiEnv("COMPlus_JitDisasmAssemblies", Path.GetFileNameWithoutExtension(assembly.Name));
AppendEnvironmentVariableToPmiEnv("COMPlus_JitUnwindDump", "*");
AppendEnvironmentVariableToPmiEnv("COMPlus_JitEHDump", "*");
AppendEnvironmentVariableToPmiEnv("DOTNET_JitDisasm", "*");
AppendEnvironmentVariableToPmiEnv("DOTNET_JitDisasmAssemblies", Path.GetFileNameWithoutExtension(assembly.Name));
AppendEnvironmentVariableToPmiEnv("DOTNET_JitUnwindDump", "*");
AppendEnvironmentVariableToPmiEnv("DOTNET_JitEHDump", "*");
if (!this._config.NoDiffable)
{
AppendEnvironmentVariableToPmiEnv("COMPlus_JitDiffableDasm", "1");
AppendEnvironmentVariableToPmiEnv("DOTNET_JitDiffableDasm", "1");
}
AppendEnvironmentVariableToPmiEnv("COMPlus_ReadyToRun", "0");
AppendEnvironmentVariableToPmiEnv("COMPlus_ZapDisable", "1");
AppendEnvironmentVariableToPmiEnv("COMPlus_JitEnableNoWayAssert", "1"); // Force noway_assert to generate assert (not fall back to MinOpts).
AppendEnvironmentVariableToPmiEnv("COMPlus_JitNoForceFallback", "1"); // Don't stress noway fallback path.
AppendEnvironmentVariableToPmiEnv("COMPlus_JitRequired", "1"); // Force NO_WAY to generate assert. Also generates assert for BADCODE/BADCODE3.
AppendEnvironmentVariableToPmiEnv("DOTNET_ReadyToRun", "0");
AppendEnvironmentVariableToPmiEnv("DOTNET_ZapDisable", "1");
AppendEnvironmentVariableToPmiEnv("DOTNET_JitEnableNoWayAssert", "1"); // Force noway_assert to generate assert (not fall back to MinOpts).
AppendEnvironmentVariableToPmiEnv("DOTNET_JitNoForceFallback", "1"); // Don't stress noway fallback path.
AppendEnvironmentVariableToPmiEnv("DOTNET_JitRequired", "1"); // Force NO_WAY to generate assert. Also generates assert for BADCODE/BADCODE3.

// We likely don't want tiering enabled, but allow it, if user wants tier0 codegen
AppendEnvironmentVariableToPmiEnv("COMPlus_TieredCompilation", _config.Tier0 ? "1" : "0");
AppendEnvironmentVariableToPmiEnv("DOTNET_TieredCompilation", _config.Tier0 ? "1" : "0");

if (_config.Tier0)
{
// jit all methods at tier0
AppendEnvironmentVariableToPmiEnv("COMPlus_TC_QuickJitForLoops", "1");
AppendEnvironmentVariableToPmiEnv("DOTNET_TC_QuickJitForLoops", "1");
// don't promote any method to tier1
AppendEnvironmentVariableToPmiEnv("COMPlus_TC_CallCounting", "0");
AppendEnvironmentVariableToPmiEnv("DOTNET_TC_CallCounting", "0");
}

if (this.doGCDump)
{
AppendEnvironmentVariableToPmiEnv("COMPlus_JitGCDump", "*");
AppendEnvironmentVariableToPmiEnv("DOTNET_JitGCDump", "*");
}

if (this.doDebugDump)
{
AppendEnvironmentVariableToPmiEnv("COMPlus_JitDebugDump", "*");
AppendEnvironmentVariableToPmiEnv("DOTNET_JitDebugDump", "*");
}

if (this._altjit != null)
{
AppendEnvironmentVariableToPmiEnv("COMPlus_AltJit", "*");
AppendEnvironmentVariableToPmiEnv("COMPlus_AltJitName", _altjit);
AppendEnvironmentVariableToPmiEnv("DOTNET_AltJit", "*");
AppendEnvironmentVariableToPmiEnv("DOTNET_AltJitName", _altjit);

const string arm64AsTarget = "_arm64_";
int targetArm64 = _altjit.IndexOf(arm64AsTarget);
Expand All @@ -541,7 +541,7 @@ void AppendEnvironmentVariableToPmiEnv(string varName, string varValue)
{
// If this looks like a cross-targeting altjit with a arm64 target and a different host
// then fix the SIMD size.
AppendEnvironmentVariableToPmiEnv("COMPlus_SIMD16ByteOnly", "1");
AppendEnvironmentVariableToPmiEnv("DOTNET_SIMD16ByteOnly", "1");
}
}
}
Expand All @@ -565,7 +565,7 @@ void AppendEnvironmentVariableToPmiEnv(string varName, string varValue)

Utility.EnsureParentDirectoryExists(dasmPath);

AppendEnvironmentVariableToPmiEnv("COMPlus_JitStdOutFile", dasmPath);
AppendEnvironmentVariableToPmiEnv("DOTNET_JitStdOutFile", dasmPath);

AddEnvironmentVariable("PMIENV", pmiEnv.ToString());

Expand Down Expand Up @@ -610,7 +610,7 @@ void AppendEnvironmentVariableToPmiEnv(string varName, string varValue)

if (hasOutput && File.Exists(logPath) && !File.Exists(dasmPath))
{
// Looks like the JIT does not support COMPlus_JitStdOutFile so
// Looks like the JIT does not support DOTNET_JitStdOutFile so
// the assembly output must be in the log file.
File.Move(logPath, dasmPath);
}
Expand Down
Loading