Skip to content

Stabilized perf results for KMeansAndLogisticRegression benchmark #10

New issue
New issue
Open
Open
Stabilized perf results for KMeansAndLogisticRegression benchmark#10
Assignees
briancylui

Description

@briancylui
briancylui
opened on Aug 30, 2018
// * Legends *
  Mean         : Arithmetic mean of all measurements
  Error        : Half of 99.9% confidence interval
  StdDev       : Standard deviation of all measurements
  Extra Metric : Value of the provided extra metric
  Gen 0        : GC Generation 0 collects per 1k Operations
  Gen 1        : GC Generation 1 collects per 1k Operations
  Gen 2        : GC Generation 2 collects per 1k Operations
  Allocated    : Allocated memory per single operation (managed only, inclusive, 1KB = 1024B)
  1 s          : 1 Second (1 sec)

netcoreapp3.0 with my change:

Iteration 1:

BenchmarkDotNet=v0.11.1, OS=Windows 10.0.17134.228 (1803/April2018Update/Redstone4)
Intel Core i7-7700 CPU 3.60GHz (Kaby Lake), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=2.2.100-refac-20180613-1
  [Host]     : .NET Core 3.0.0-preview1-26822-01 (CoreCLR 4.6.26817.05, CoreFX 4.6.26817.04), 64bit RyuJIT
  Job-YPIYKX : .NET Core 3.0.0-preview1-26822-01 (CoreCLR 4.6.26817.05, CoreFX 4.6.26817.04), 64bit RyuJIT

Toolchain=netcoreapp3.0  MaxIterationCount=20  WarmupCount=1
Method Mean Error StdDev Extra Metric Gen 0 Gen 1 Gen 2 Allocated
TrainKMeansAndLR 1.781 s 0.0976 s 0.1044 s - 16000.0000 8000.0000 3000.0000 47.57 MB

Iteration 2:

BenchmarkDotNet=v0.11.1, OS=Windows 10.0.17134.228 (1803/April2018Update/Redstone4)
Intel Core i7-7700 CPU 3.60GHz (Max: 1.90GHz) (Kaby Lake), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=2.2.100-refac-20180613-1
  [Host]     : .NET Core 3.0.0-preview1-26822-01 (CoreCLR 4.6.26817.05, CoreFX 4.6.26817.04), 64bit RyuJIT
  Job-GREASD : .NET Core 3.0.0-preview1-26822-01 (CoreCLR 4.6.26817.05, CoreFX 4.6.26817.04), 64bit RyuJIT

Toolchain=netcoreapp3.0  MaxIterationCount=20  WarmupCount=1
Method Mean Error StdDev Extra Metric Gen 0 Gen 1 Gen 2 Allocated
TrainKMeansAndLR 1.781 s 0.1050 s 0.1209 s - 16000.0000 8000.0000 3000.0000 47.27 MB

Iteration 3:

BenchmarkDotNet=v0.11.1, OS=Windows 10.0.17134.228 (1803/April2018Update/Redstone4)
Intel Core i7-7700 CPU 3.60GHz (Kaby Lake), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=2.2.100-refac-20180613-1
  [Host]     : .NET Core 3.0.0-preview1-26822-01 (CoreCLR 4.6.26817.05, CoreFX 4.6.26817.04), 64bit RyuJIT
  Job-OHSMOJ : .NET Core 3.0.0-preview1-26822-01 (CoreCLR 4.6.26817.05, CoreFX 4.6.26817.04), 64bit RyuJIT

Toolchain=netcoreapp3.0  MaxIterationCount=20  WarmupCount=1
Method Mean Error StdDev Extra Metric Gen 0 Gen 1 Gen 2 Allocated
TrainKMeansAndLR 1.853 s 0.0926 s 0.1067 s - 16000.0000 8000.0000 3000.0000 47.17 MB

Iteration 4:

BenchmarkDotNet=v0.11.1, OS=Windows 10.0.17134.228 (1803/April2018Update/Redstone4)
Intel Core i7-7700 CPU 3.60GHz (Kaby Lake), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=2.2.100-refac-20180613-1
  [Host]     : .NET Core 3.0.0-preview1-26822-01 (CoreCLR 4.6.26817.05, CoreFX 4.6.26817.04), 64bit RyuJIT
  Job-MIAXEM : .NET Core 3.0.0-preview1-26822-01 (CoreCLR 4.6.26817.05, CoreFX 4.6.26817.04), 64bit RyuJIT

Toolchain=netcoreapp3.0  MaxIterationCount=20  WarmupCount=1
Method Mean Error StdDev Extra Metric Gen 0 Gen 1 Gen 2 Allocated
TrainKMeansAndLR 1.797 s 0.1104 s 0.1271 s - 16000.0000 8000.0000 3000.0000 47.27 MB

netcoreapp3.0 without my change:

Iteration 1:

BenchmarkDotNet=v0.11.1, OS=Windows 10.0.17134.228 (1803/April2018Update/Redstone4)
Intel Core i7-7700 CPU 3.60GHz (Kaby Lake), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=2.2.100-refac-20180613-1
  [Host]     : .NET Core 3.0.0-preview1-26822-01 (CoreCLR 4.6.26817.05, CoreFX 4.6.26817.04), 64bit RyuJIT
  Job-TIGZKX : .NET Core 3.0.0-preview1-26822-01 (CoreCLR 4.6.26817.05, CoreFX 4.6.26817.04), 64bit RyuJIT

Toolchain=netcoreapp3.0  MaxIterationCount=20  WarmupCount=1
Method Mean Error StdDev Extra Metric Gen 0 Gen 1 Gen 2 Allocated
TrainKMeansAndLR 2.094 s 0.0440 s 0.0489 s - 17000.0000 8000.0000 3000.0000 47.13 MB

Iteration 2:

BenchmarkDotNet=v0.11.1, OS=Windows 10.0.17134.228 (1803/April2018Update/Redstone4)
Intel Core i7-7700 CPU 3.60GHz (Max: 1.90GHz) (Kaby Lake), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=2.2.100-refac-20180613-1
  [Host]     : .NET Core 3.0.0-preview1-26822-01 (CoreCLR 4.6.26817.05, CoreFX 4.6.26817.04), 64bit RyuJIT
  Job-LMZGNT : .NET Core 3.0.0-preview1-26822-01 (CoreCLR 4.6.26817.05, CoreFX 4.6.26817.04), 64bit RyuJIT

Toolchain=netcoreapp3.0  MaxIterationCount=20  WarmupCount=1
Method Mean Error StdDev Extra Metric Gen 0 Gen 1 Gen 2 Allocated
TrainKMeansAndLR 2.062 s 0.0828 s 0.0920 s - 17000.0000 8000.0000 3000.0000 47.17 MB

Iteration 3:

BenchmarkDotNet=v0.11.1, OS=Windows 10.0.17134.228 (1803/April2018Update/Redstone4)
Intel Core i7-7700 CPU 3.60GHz (Kaby Lake), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=2.2.100-refac-20180613-1
  [Host]     : .NET Core 3.0.0-preview1-26822-01 (CoreCLR 4.6.26817.05, CoreFX 4.6.26817.04), 64bit RyuJIT
  Job-NAKODA : .NET Core 3.0.0-preview1-26822-01 (CoreCLR 4.6.26817.05, CoreFX 4.6.26817.04), 64bit RyuJIT

Toolchain=netcoreapp3.0  MaxIterationCount=20  WarmupCount=1
Method Mean Error StdDev Extra Metric Gen 0 Gen 1 Gen 2 Allocated
TrainKMeansAndLR 2.139 s 0.0677 s 0.0780 s - 17000.0000 9000.0000 3000.0000 47.12 MB

Iteration 4:

BenchmarkDotNet=v0.11.1, OS=Windows 10.0.17134.228 (1803/April2018Update/Redstone4)
Intel Core i7-7700 CPU 3.60GHz (Kaby Lake), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=2.2.100-refac-20180613-1
  [Host]     : .NET Core 3.0.0-preview1-26822-01 (CoreCLR 4.6.26817.05, CoreFX 4.6.26817.04), 64bit RyuJIT
  Job-QVPJCB : .NET Core 3.0.0-preview1-26822-01 (CoreCLR 4.6.26817.05, CoreFX 4.6.26817.04), 64bit RyuJIT

Toolchain=netcoreapp3.0  MaxIterationCount=20  WarmupCount=1
Method Mean Error StdDev Extra Metric Gen 0 Gen 1 Gen 2 Allocated
TrainKMeansAndLR 2.079 s 0.0778 s 0.0896 s - 17000.0000 9000.0000 3000.0000 47.36 MB

Activity

  1. briancylui commented on Aug 30, 2018

    @briancylui
    briancylui
    on Aug 30, 2018
    OwnerAuthor

    @danmosemsft @eerhardt @safern @tannergooding: In netcoreapp3.0, perf results with my change are at least 10% better than those without my change. Will present these.

  2. briancylui
    self-assigned this
    on Aug 30, 2018

  3. eerhardt commented on Aug 31, 2018

    @eerhardt
    eerhardt
    on Aug 31, 2018
    Collaborator

    This is awesome results, @briancylui! I'm excited to see the presentation tomorrow.

  4. safern commented on Aug 31, 2018

    @safern
    safern
    on Aug 31, 2018

    This is great!! I love the fact that ML.NET can be used in different architectures, by building on one of them and getting perf benefits!

  5. helloguo commented on Sep 25, 2018

    @helloguo
    helloguo
    on Sep 25, 2018

    @briancylui can you share how to build and run ML.NET and Benchmark KMeansAndLogisticRegression with .NET Core 3.0 with your change?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions