Unable to detect anomaly in data that appears anomalous

System information

OS version/distro:
.NET Version (eg., dotnet --info):

Issue

What did you do?
Attempted to detect anomalies on time series data
What happened?
No anomalies were detected
What did you expect?
An anomaly on either the 1st or 2nd data point

Source code / logs

Sensitivity=100, SeasonalPeriodForAnomalyDetection=-1 (we set it to 0 when it is < 0), Threshold=0.1

Please paste or attach the code or logs or traces that would be helpful to diagnose the issue you are reporting.

` [Fact]
public void TestRunSrCnnAnomalyDetector()
{
var ml = new MLContext(1);
IDataView dataView;
List data = new List();

        var dataPath = @"PathToInputFile";
        var outputPath = @"PathToOutputFile";
        var valueColumnIndex = 1;

        // Load data from file into the dataView

        using StreamReader inputFile = new StreamReader(dataPath);
        var line = inputFile.ReadLine();
        while((line = inputFile.ReadLine()) != null)
        {
            var content = line.Split(",".ToCharArray());
            data.Add(new TimeSeriesDataDouble { Value=Double.Parse(content[valueColumnIndex]) });
        }
        dataView = ml.Data.LoadFromEnumerable<TimeSeriesDataDouble>(data);

        // Setup the detection arguments
        string outputColumnName = nameof(SrCnnAnomalyDetection.Prediction);
        string inputColumnName = nameof(TimeSeriesDataDouble.Value);

        // Do batch anomaly detection
        var options = new SrCnnEntireAnomalyDetectorOptions()
        {
            Threshold = 0.23,
            BatchSize = -1,
            Sensitivity = 53.0,
            DetectMode = SrCnnDetectMode.AnomalyAndMargin,
            Period = 0,
            DeseasonalityMode = SrCnnDeseasonalityMode.Stl
        };

        var outputDataView = ml.AnomalyDetection.DetectEntireAnomalyBySrCnn(dataView, outputColumnName, inputColumnName, options);

        // Getting the data of the newly created column as an IEnumerable of SrCnnAnomalyDetection.
        var predictionColumn = ml.Data.CreateEnumerable<SrCnnAnomalyDetection>(
            outputDataView, reuseRowObject: false);

        using StreamWriter file = new StreamWriter(outputPath, false);
        file.WriteLine("value,isAnomaly,anomalyScore,magnitude,expectedValue,boundaryUnits,upperBoundary,lowerBoundary");
        var k = 0;
        foreach (var prediction in predictionColumn)
        {
            file.WriteLine("{0},{1},{2},{3},{4},{5},{6},{7}", data[k].Value, 
                prediction.Prediction[0], prediction.Prediction[1], prediction.Prediction[2], prediction.Prediction[3], prediction.Prediction[4], prediction.Prediction[5], prediction.Prediction[6]);
            ++k;
        }
    }

`
Could you use this script to run the case and update the output here?

Here's the output I got. Looks like no anomalies were detected

value,isAnomaly,anomalyScore,magnitude,expectedValue,boundaryUnits,upperBoundary,lowerBoundary
181.944,0,0,0.522966128746679,177.443091674641,103.145815789474,363.105560095693,-8.21937674641117
37.176,0,0,0.745155271671293,41.731366507177,104.393815789474,229.640234928229,-146.177501913875
57.14,0,0,0.136707235856698,52.5301753110048,105.641815789474,242.685443732057,-137.625093110047
67.128,0,0,0.0850154360612621,71.7922828708134,106.889815789474,264.193951291866,-120.609385550239
72.12,0,0,0.0591133826385268,67.9172361171339,108.137815789474,262.565304538186,-126.730832303918
77.112,0,0,0.0404630444656698,76.9609900863512,108.386815789474,272.057258507403,-118.135278334701
82.104,0,0,0.0176426322801254,84.9100212024565,108.635815789474,280.454489623509,-110.634447218596
83.1,0,0,0.0407285436916503,79.703903546285,108.635815789474,275.248371967337,-115.840564874767
87.09,0,0,0.0240877864738463,89.4251290248719,110.630815789474,288.560597445924,-109.71033939618
92.088,0,0,0.00958621604240923,91.5221811402331,113.090815789474,295.085649561285,-112.041287280819
92.01,0,0,0.0441215707348716,91.1346791375452,113.129815789474,294.768347558597,-112.498989283507
97.08,0,0,0.0295952132892941,98.36993909615,115.625815789474,306.496407517202,-109.756529324902
102.072,0,0,0.0163203092514839,101.501352193267,118.121815789474,314.120620614319,-111.117916227785
107.05,0,0,0.02164138229226,106.231499044153,120.610815789474,323.330967465205,-110.8679693769
107.06,0,0,0.0592867107998528,109.168240468027,120.615815789474,326.27670888908,-107.940227953025
117.048,0,0,0.0295676779099656,114.440282315818,125.609815789474,340.53795073687,-111.657386105234
122.04,0,0,0.0331004407058219,124.071088970008,128.105815789474,354.661557391061,-106.519379451044
132.024,0,0,0.0312363029168756,131.400139994489,133.097815789474,370.976208415541,-108.175928426564
147,0,0,0.0775118579077661,146.038275536252,140.585815789474,399.092743957304,-107.016192884801
151.82,0,0,0.0598045713810684,153.80015468422,142.945815789474,411.102623105272,-103.502313736832
151.992,0,0,0.0454796096199837,150.042847907984,142.995815789474,407.435316329036,-107.349620513068
151.72,0,0,0.0345565564469605,152.604906834188,143.055815789474,410.10537525524,-104.895561586864
151.94,0,0,0.0165437153609649,152.633578289076,143.081815789474,410.180846710128,-104.913690131976
156.969,0,0,0.0514029138377458,154.972039905732,145.545815789474,416.954508326785,-107.01042851532
156.984,0,0,0.0371400702625073,159.317415743382,145.570315789474,421.343984164434,-102.70915267767
156.92,0,0,0.0265062296857799,155.473270112141,145.577815789474,417.513338533193,-106.566798308911
161.976,0,0,0.0563217473605611,161.643982420086,148.055815789474,428.144450841138,-104.856486000966
161.94,0,0,0.0288975265968351,164.137459420599,148.070815789474,430.664927841652,-102.390009000453
161.97,0,0,0.0193578473815232,158.75933018877,148.073815789474,425.292198609822,-107.773538232282
166.968,0,0,0.0251727532424213,169.69585533932,150.569815789474,440.721523760372,-101.329813081732
176.952,0,0,0.0678386468339108,176.220076509857,155.561815789474,456.231344930909,-103.791191911195
181.94,0,0,0.0515876270252644,179.855132041855,158.055815789474,464.355600462907,-104.645336379197
186.936,0,0,0.0302207536901738,191.475152683818,159.304815789474,478.22382110487,-95.2735157372339
201.91,0,0,0.134792954830084,196.508096906239,160.553815789474,485.504965327291,-92.4887715148131
201.912,0,0,0.0794219555743357,205.810422687736,164.294815789474,501.541091108788,-89.9202457333166
201.9,0,0,0.056394182910596,201.839752928706,168.035815789474,504.304221349759,-100.624715492346
206.904,0,0,0.0624443233190773,201.683491536713,168.038315789474,504.152459957765,-100.785476884339
216.88,0,0,0.152018791404235,224.212690143541,168.040815789474,526.686158564593,-78.2607782775114

The root cause lies in the SR score function, which is
score = abs(mag - ave_mag) / ave_mag.

For the first point, ave_mag = mag, then
score_0 = (mag_0 - mag_0) / mag_0 = 0

For the second point, ave_mag = (mag_0 + mag_1) / 2, then
score_1 = (mag_1 - (mag_0 + mag_1) / 2) / ((mag_0 + mag_1) / 2) = abs(mag_1 - mag_0) / (mag_0 + mag_1) < 1.

So both two can not ever become anomaly points.

We proposed the solution to use the ave_mag of first full window to replace the avg_mag of first W points, where W is the score window size.

The output after the fix is attached below. with parameters


            var options = new SrCnnEntireAnomalyDetectorOptions()
            {
                Threshold = 0.20,
                BatchSize = -1,
                Sensitivity = 99.0,
                DetectMode = SrCnnDetectMode.AnomalyAndMargin,
                Period = 0,
                DeseasonalityMode = SrCnnDeseasonalityMode.Stl
            };


value | isAnomaly | anomalyScore | magnitude | expectedValue | boundaryUnits | upperBoundary | lowerBoundary
181.944 | 1 | 0.075261 | 0.522966 | 187.8224 | 104.8348 | 187.8538 | 187.7909
37.176 | 1 | 0.402395 | 0.745155 | 123.0586 | 106.0828 | 123.0905 | 123.0268
57.14 | 0 | 0 | 0.136707 | 61.82353 | 107.3308 | 61.85573 | 61.79133
67.128 | 0 | 0 | 0.085015 | 62.39002 | 108.5788 | 62.42259 | 62.35744
72.12 | 0 | 0 | 0.059113 | 75.75983 | 109.8268 | 75.79277 | 75.72688
77.112 | 0 | 0 | 0.040463 | 75.72429 | 110.0758 | 75.75731 | 75.69127
82.104 | 0 | 0 | 0.017643 | 81.58817 | 110.3248 | 81.62127 | 81.55507
83.1 | 0 | 0 | 0.040729 | 84.67297 | 110.3248 | 84.70607 | 84.63988
87.09 | 0 | 0 | 0.024088 | 85.50223 | 112.3198 | 85.53592 | 85.46853
92.088 | 0 | 0 | 0.009586 | 92.8119 | 114.7798 | 92.84633 | 92.77746
92.01 | 0 | 0 | 0.044122 | 92.58165 | 114.8188 | 92.61609 | 92.5472
97.08 | 0 | 0 | 0.029595 | 95.33322 | 117.3148 | 95.36842 | 95.29803
102.072 | 0 | 0 | 0.01632 | 104.4254 | 119.8108 | 104.4613 | 104.3895
107.05 | 0 | 0 | 0.021641 | 104.8445 | 122.2998 | 104.8812 | 104.8078
107.06 | 0 | 0 | 0.059287 | 108.4873 | 122.3048 | 108.524 | 108.4506
117.048 | 0 | 0 | 0.029568 | 116.6705 | 127.2988 | 116.7087 | 116.6323
122.04 | 0 | 0 | 0.0331 | 121.5268 | 129.7948 | 121.5658 | 121.4879
132.024 | 0 | 0 | 0.031236 | 132.9459 | 134.7868 | 132.9864 | 132.9055
147 | 0 | 0 | 0.077512 | 146.2425 | 142.2748 | 146.2852 | 146.1998
151.82 | 0 | 0 | 0.059805 | 151.9991 | 144.6348 | 152.0425 | 151.9557
151.992 | 0 | 0 | 0.04548 | 152.4825 | 144.6848 | 152.5259 | 152.4391
151.72 | 0 | 0 | 0.034557 | 150.8038 | 144.7448 | 150.8473 | 150.7604
151.94 | 0 | 0 | 0.016544 | 152.8378 | 144.7708 | 152.8813 | 152.7944
156.969 | 0 | 0 | 0.051403 | 156.5178 | 147.2348 | 156.562 | 156.4736
156.984 | 0 | 0 | 0.03714 | 156.7732 | 147.2593 | 156.8173 | 156.729
156.92 | 0 | 0 | 0.026506 | 157.7035 | 147.2668 | 157.7477 | 157.6593
161.976 | 0 | 0 | 0.056322 | 160.963 | 149.7448 | 161.0079 | 160.9181
161.94 | 0 | 0 | 0.028898 | 162.7505 | 149.7598 | 162.7954 | 162.7055
161.97 | 0 | 0 | 0.019358 | 161.6834 | 149.7628 | 161.7283 | 161.6384
166.968 | 0 | 0 | 0.025173 | 166.6591 | 152.2588 | 166.7048 | 166.6135
176.952 | 0 | 0 | 0.067839 | 177.667 | 157.2508 | 177.7142 | 177.6199
181.94 | 0 | 0 | 0.051588 | 181.1448 | 159.7448 | 181.1928 | 181.0969
186.936 | 0 | 0 | 0.030221 | 187.5522 | 160.9938 | 187.6005 | 187.504
201.91 | 0 | 0 | 0.134793 | 201.4772 | 162.2428 | 201.5258 | 201.4285
201.912 | 0 | 0 | 0.079422 | 202.4886 | 165.9838 | 202.5384 | 202.4388
201.9 | 0 | 0 | 0.056394 | 200.6031 | 169.7248 | 200.654 | 200.5521
206.904 | 0 | 0 | 0.062444 | 209.5261 | 169.7273 | 209.577 | 209.4752
216.88 | 0 | 0 | 0.152019 | 212.5762 | 169.7298 | 212.6271 | 212.5252

@guinao

It seems this got fixed in #5502 . Thanks @guinao for fixing it!

Navigation Menu

Repository navigation

Unable to detect anomaly in data that appears anomalous #5491

System information

Issue

Source code / logs

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Unable to detect anomaly in data that appears anomalous #5491

Description

System information

Issue

Source code / logs

Activity

guinao commented on Nov 18, 2020

nnoradie commented on Nov 18, 2020

guinao commented on Nov 20, 2020

guinao commented on Nov 20, 2020

antoniovs1029 commented on Dec 28, 2020

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions