Files
sqltoolsservice/test/Microsoft.InsightsGenerator.UnitTests/SignatureGeneratorTests.cs
Karl Burtram 5cf5b59a0d Insights generator project (#1066)
* InsightsGenerator project template files

* Add insights projects to SLN

* Setting up siggen class (#1003)

* Setting up siggen class

* fixed the learn method

* Refactoring code
Fixed compile errors

* renamed results to result

* Basic transformation logic (#1004)

* Fix a couple bugs and add a simple test (#1007)

* Fix a couple bugs and add a simple test

* More tests and bug fix

* Nara/workflow (#1006)

* added a queue processor

* ordered  using statements

* Armemon/analytics (#1008)

* Basic transformation logic

* changed some structure of siggen

* added sum and average method, as well as select rows by input name

* add insights to results

* min, max added

Co-authored-by: Karl Burtram <karlb@microsoft.com>
Co-authored-by: Aasim Khan <aasimkhan30@gmail.com>
Co-authored-by: Arslan Memon <armemon@microsoft.com>

* Added rules engine base implementation (#1005)

* Added rules engine base implementation

* update comments

* addressing comments

* adding template text to columnheaders object

* adding template text to columnheaders object

* fixing columnheaders class

* Added test

* Added Template Parser unit test in Test project

* Deleted unnecessary files and reverted the files that were modified by mistake

Co-authored-by: Jinjing Arima <jiarima@microsoft.com>

* Insights generator message handler placeholder (#1013)

* Aasim/insights/insight methods (#1014)

* Basic transformation logic

* changed some structure of siggen

* Added top and bottom insight functions

* Added top, bottom insights
Added tests for top, bottom insights

* Armemon/insights2 (#1011)

* max and min insightsperslice, and tests

* got rid of unneccesssary function

* get indexes

Co-authored-by: Arslan Memon <armemon@microsoft.com>

* Armemon/insights2 (#1012)

* max and min insightsperslice, and tests

* got rid of unneccesssary function

* get indexes

* learn for stringinputtyype

* add learn implentation

Co-authored-by: Arslan Memon <armemon@microsoft.com>

* Added Tests
Removed duplicate methods

Co-authored-by: Karl Burtram <karlb@microsoft.com>
Co-authored-by: arslan9955 <53170027+arslan9955@users.noreply.github.com>
Co-authored-by: Arslan Memon <armemon@microsoft.com>

* Armemon/insights2 (#1016)

* Basic transformation logic

* changed some structure of siggen

* Added top and bottom insight functions

* Added top, bottom insights
Added tests for top, bottom insights

* max and min insightsperslice, and tests

* got rid of unneccesssary function

* get indexes

* learn for stringinputtyype

* add learn implentation

* add unique inputs

* fix merge error

* add to result

Co-authored-by: Karl Burtram <karlb@microsoft.com>
Co-authored-by: Aasim Khan <aasimkhan30@gmail.com>
Co-authored-by: Arslan Memon <armemon@microsoft.com>

* Added all the templates (#1015)

* Added all the templates
Added a method to find matched template

* Added a function to replace # and ## values in a template

* Added ReplaceHashesInTemplate call

* Added comments

* Updated the template txt

* Updated GetTopHeadersWithHash function to add #toplist

* Updated the logic per offline discussion with Hermineh

* Update request handler contract to take array (#1020)

* added rulesengine findmatchingtemplate (#1019)

* Add support for getting DacFx deploy options from a publish profile (#995)

* add support for getting options from a publish profile

* update comments

* set values for default options if they aren't specified in the publish profile

* addressing comments

* Updating to latest DacFx for a bug fix (#1010)

* added rulesengine findmatchingtemplate

* Update DacFx deploy and generate script with options (#998)

* update deploy and generate script to accept deployment options

* add tests

* add test with option set to true

* merge

* merge

* incorporated FindMatchedTemplate

Co-authored-by: Kim Santiago <31145923+kisantia@users.noreply.github.com>
Co-authored-by: Udeesha Gautam <46980425+udeeshagautam@users.noreply.github.com>

* -Added logic for Insights Generator Service Handler (#1017)

* -Added logic for Insights Generator Service Handler

* Fixed some logic

* Adding workflow test

* Update transform and add tests (#1024)

* Jiarima/fix rules engine logic (#1021)

* Added all the templates
Added a method to find matched template

* Added a function to replace # and ## values in a template

* Added ReplaceHashesInTemplate call

* Added comments

* Updated the template txt

* Updated GetTopHeadersWithHash function to add #toplist

* Updated the logic per offline discussion with Hermineh

* Update with the fixes

* Updated template and foreach conditions

* Added distinct

* Updated tests according to the logic change (#1026)

* Nara/remove queing (#1023)

* loc update (#914)

* loc update

* loc updates

* Add support for getting DacFx deploy options from a publish profile (#995)

* add support for getting options from a publish profile

* update comments

* set values for default options if they aren't specified in the publish profile

* addressing comments

* Updating to latest DacFx for a bug fix (#1010)

* Update DacFx deploy and generate script with options (#998)

* update deploy and generate script to accept deployment options

* add tests

* add test with option set to true

* intermediate check in for merge, transformed not working

* intermediate check in for merge, transformed not working

* added test case

* merged

Co-authored-by: khoiph1 <khoiph@microsoft.com>
Co-authored-by: Kim Santiago <31145923+kisantia@users.noreply.github.com>
Co-authored-by: Udeesha Gautam <46980425+udeeshagautam@users.noreply.github.com>

* Output data types from transform (#1029)

* Fix bug process input_g (#1030)

* Fixed the insight generator service (#1028)

* Jiarima/added more testings (#1031)

* Added another test
Updated ReplaceHashesInTemplate function to return string instead of Template

* Added third test

* Merged

* Reverted the workflow file to match with the one in hack/insights

* Bugs fixes to hook insights up to ADS (#1033)

* Bug fixes for hack insights (#1032)

* Fixed the minColumn index bug in Data Transformation
Fixed the template matching logic.

* Adding changes from PR

* Try to fix Workflow tests

* Readd workflow tests

* Fix template load location

Co-authored-by: Aasim Khan <aasimkhan30@gmail.com>
Co-authored-by: Nara <NaraVen@users.noreply.github.com>
Co-authored-by: arslan9955 <53170027+arslan9955@users.noreply.github.com>
Co-authored-by: Arslan Memon <armemon@microsoft.com>
Co-authored-by: gadudhbh <68879970+gadudhbh@users.noreply.github.com>
Co-authored-by: Jinjing Arima <jiarima@microsoft.com>
Co-authored-by: jiarima <68882862+jiarima@users.noreply.github.com>
Co-authored-by: Kim Santiago <31145923+kisantia@users.noreply.github.com>
Co-authored-by: Udeesha Gautam <46980425+udeeshagautam@users.noreply.github.com>
Co-authored-by: khoiph1 <khoiph@microsoft.com>
2020-09-03 14:15:51 -07:00

340 lines
8.3 KiB
C#

using System;
using System.Collections.Generic;
using System.Text;
using Xunit;
namespace Microsoft.InsightsGenerator.UnitTests
{
public class SignatureGeneratorTests
{
[Fact]
public void TopTest()
{
var expectedTopInsight = @"top
3
China (455) 19.73%
Turkey (254) 11.01%
United States (188) 8.15%";
SignatureGenerator sigGen = new SignatureGenerator(sampleDataArray(false));
sigGen.OverallTopInsights(3, 0, 1);
CompareInsightWithExpectedOutcome(sigGen.Result.Insights, expectedTopInsight);
}
[Fact]
public void TopSliceTest()
{
var expectedTopSliceInsight = @"topPerSlice
5
Category1
3
China (455) 34.89%
Turkey (254) 19.48%
United States (188) 14.42%
Category2
3
Japan (171) 91.94%
China (10) 5.38%
United States (3) 1.61%
Category3
3
United States (106) 15.5%
Brazil (91) 13.3%
Korea (61) 8.92%
Category4
3
United States (38) 38%
China (12) 12%
Korea (8) 8%
Category5
3
Korea (21) 65.62%
United States (6) 18.75%
Canada (3) 9.38%";
SignatureGenerator sigGen = new SignatureGenerator(sampleDataArray(false));
sigGen.SlicedTopInsights(3, 0, 2, 1);
CompareInsightWithExpectedOutcome(sigGen.Result.Insights, expectedTopSliceInsight);
}
[Fact]
public void BottomTest()
{
var expectedBottomInsight = @"bottom
3
Korea (1) 0.04%
Germany (1) 0.04%
India (1) 0.04%";
SignatureGenerator sigGen = new SignatureGenerator(sampleDataArray(false));
sigGen.OverallBottomInsights(3, 0, 1);
CompareInsightWithExpectedOutcome(sigGen.Result.Insights, expectedBottomInsight);
}
[Fact]
public void BottomSliceTest()
{
var expectedBottomSliceInsight = @"bottomPerSlice
5
Category1
3
Canada (17) 1.3%
United Kingdom (17) 1.3%
Vietnam (18) 1.38%
Category2
3
Germany (1) 0.54%
Korea (1) 0.54%
United States (3) 1.61%
Category3
3
France (12) 1.75%
United Kingdom (20) 2.92%
Vietnam (22) 3.22%
Category4
3
India (1) 1%
Japan (1) 1%
Canada (2) 2%
Category5
3
India (1) 3.12%
Japan (1) 3.12%
Canada (3) 9.38%";
SignatureGenerator sigGen = new SignatureGenerator(sampleDataArray(false));
sigGen.SlicedBottomInsights(3, 0, 2, 1);
CompareInsightWithExpectedOutcome(sigGen.Result.Insights, expectedBottomSliceInsight);
}
[Fact]
public void AverageTest()
{
var expectedAverageInsight = @"average
42.7";
SignatureGenerator sigGen = new SignatureGenerator(sampleDataArray(false));
sigGen.OverallAverageInsights(1);
CompareInsightWithExpectedOutcome(sigGen.Result.Insights, expectedAverageInsight);
}
[Fact]
public void SumTest()
{
var expectedSumInsight = @"sum
2306";
SignatureGenerator sigGen = new SignatureGenerator(sampleDataArray(false));
sigGen.OverallSumInsights(1);
CompareInsightWithExpectedOutcome(sigGen.Result.Insights, expectedSumInsight);
}
[Fact]
public void SlicedSumTest()
{
var expectedSlicedSumInsight = @"sumPerSlice
5
Category1
1304
Category2
186
Category3
684
Category4
100
Category5
32";
SignatureGenerator sigGen = new SignatureGenerator(sampleDataArray(false));
sigGen.SlicedSumInsights(2, 1);
CompareInsightWithExpectedOutcome(sigGen.Result.Insights, expectedSlicedSumInsight);
}
[Fact]
public void SlicedAverageTest()
{
var expectedSlicedAverageInsight = @"sumPerSlice
5
Category1
86.93
Category2
37.2
Category3
45.6
Category4
7.14
Category5
6.4";
SignatureGenerator sigGen = new SignatureGenerator(sampleDataArray(false));
sigGen.SlicedAverageInsights(2, 1);
CompareInsightWithExpectedOutcome(sigGen.Result.Insights, expectedSlicedAverageInsight);
}
[Fact]
public void SlicedPercentageTest()
{
var expectedSlicedPercentageInsight = @"percentagePerSlice
5
Category1
56.55
Category2
8.07
Category3
29.66
Category4
4.34
Category5
1.39";
SignatureGenerator sigGen = new SignatureGenerator(sampleDataArray(false));
sigGen.SlicedPercentageInsights(2, 1);
CompareInsightWithExpectedOutcome(sigGen.Result.Insights, expectedSlicedPercentageInsight);
}
[Fact]
public void MaxAndMinInsightsTest()
{
var expectedMaxAndMinInsight = @"max
455
min
1";
SignatureGenerator sigGen = new SignatureGenerator(sampleDataArray(false));
sigGen.OverallMaxInsights(1);
sigGen.OverallMinInsights(1);
CompareInsightWithExpectedOutcome(sigGen.Result.Insights, expectedMaxAndMinInsight);
}
[Fact]
public void MaxAndMinSlicedInsightsTest()
{
string expectedMaxAndMinSlicedInsight = @"maxPerSlice
5
Category1
455
Category2
171
Category3
106
Category4
38
Category5
21
minPerSlice
5
Category1
17
Category2
1
Category3
12
Category4
1
Category5
1";
SignatureGenerator sigGen = new SignatureGenerator(sampleDataArray(false));
sigGen.SlicedMaxInsights(2, 1);
sigGen.SlicedMinInsights(2, 1);
CompareInsightWithExpectedOutcome(sigGen.Result.Insights, expectedMaxAndMinSlicedInsight);
}
public void CompareInsightWithExpectedOutcome(List<List<string>> insights, string expectedOutcome)
{
List<string> stringedInsights = new List<string>();
foreach (List<string> insight in insights)
{
stringedInsights.Add(string.Join(Environment.NewLine, insight));
}
Assert.Equal(expectedOutcome, string.Join(Environment.NewLine, stringedInsights));
}
[Fact]
public void LearnTest()
{
SignatureGenerator sigGen = new SignatureGenerator(sampleDataArray(false));
sigGen.Learn();
foreach (List<string> list in sigGen.Result.Insights)
{
foreach (string str in list)
{
Console.WriteLine(str);
}
}
}
public DataArray sampleDataArray(bool timeinput)
{
DataArray sample = new DataArray();
var inputDataType = DataArray.DataType.String;
if (timeinput)
{
inputDataType = DataArray.DataType.DateTime;
}
sample.ColumnNames = new string[] { "input_g_0", "output_0", "slicer_0" };
sample.ColumnDataType = new DataArray.DataType[] { inputDataType, DataArray.DataType.Number, DataArray.DataType.String };
string sampleTableString =
@"China 455 Category1
Turkey 254 Category1
United States 188 Category1
Japan 171 Category2
United States 106 Category3
Brazil 91 Category3
Thailand 67 Category1
Korea 61 Category3
Russia 61 Category1
China 60 Category3
Brazil 57 Category1
Germany 51 Category3
Turkey 49 Category3
Russia 45 Category3
Japan 44 Category3
United States 38 Category4
Thailand 37 Category3
India 36 Category3
Germany 35 Category1
France 33 Category1
India 31 Category1
Japan 28 Category1
Mexico 27 Category3
Canada 23 Category3
Mexico 22 Category1
Vietnam 22 Category3
Korea 21 Category1
Korea 21 Category5
United Kingdom 20 Category3
Vietnam 18 Category1
Canada 17 Category1
United Kingdom 17 Category1
China 12 Category4
France 12 Category3
China 10 Category2
Korea 8 Category4
Brazil 6 Category4
Russia 6 Category4
United States 6 Category5
France 5 Category4
Germany 5 Category4
United Kingdom 5 Category4
Thailand 4 Category4
Turkey 4 Category4
Canada 3 Category5
Mexico 3 Category4
United States 3 Category2
Canada 2 Category4
Germany 1 Category2
India 1 Category4
India 1 Category5
Japan 1 Category4
Japan 1 Category5
Korea 1 Category2";
string[] sampleRows = sampleTableString.Split(Environment.NewLine);
List<string[]> sampleRowList = new List<string[]>();
foreach (var row in sampleRows)
{
sampleRowList.Add(row.Split(" "));
}
var columnTypes = new string[] { "input_g_1", "output_1", "slicer_1" };
sample.Cells = sampleRowList.ToArray();
sample.TransformedColumnNames = columnTypes;
return sample;
}
}
}