Insights generator project (#1066)

* InsightsGenerator project template files * Add insights projects to SLN * Setting up siggen class (#1003) * Setting up siggen class * fixed the learn method * Refactoring code Fixed compile errors * renamed results to result * Basic transformation logic (#1004) * Fix a couple bugs and add a simple test (#1007) * Fix a couple bugs and add a simple test * More tests and bug fix * Nara/workflow (#1006) * added a queue processor * ordered using statements * Armemon/analytics (#1008) * Basic transformation logic * changed some structure of siggen * added sum and average method, as well as select rows by input name * add insights to results * min, max added Co-authored-by: Karl Burtram <karlb@microsoft.com> Co-authored-by: Aasim Khan <aasimkhan30@gmail.com> Co-authored-by: Arslan Memon <armemon@microsoft.com> * Added rules engine base implementation (#1005) * Added rules engine base implementation * update comments * addressing comments * adding template text to columnheaders object * adding template text to columnheaders object * fixing columnheaders class * Added test * Added Template Parser unit test in Test project * Deleted unnecessary files and reverted the files that were modified by mistake Co-authored-by: Jinjing Arima <jiarima@microsoft.com> * Insights generator message handler placeholder (#1013) * Aasim/insights/insight methods (#1014) * Basic transformation logic * changed some structure of siggen * Added top and bottom insight functions * Added top, bottom insights Added tests for top, bottom insights * Armemon/insights2 (#1011) * max and min insightsperslice, and tests * got rid of unneccesssary function * get indexes Co-authored-by: Arslan Memon <armemon@microsoft.com> * Armemon/insights2 (#1012) * max and min insightsperslice, and tests * got rid of unneccesssary function * get indexes * learn for stringinputtyype * add learn implentation Co-authored-by: Arslan Memon <armemon@microsoft.com> * Added Tests Removed duplicate methods Co-authored-by: Karl Burtram <karlb@microsoft.com> Co-authored-by: arslan9955 <53170027+arslan9955@users.noreply.github.com> Co-authored-by: Arslan Memon <armemon@microsoft.com> * Armemon/insights2 (#1016) * Basic transformation logic * changed some structure of siggen * Added top and bottom insight functions * Added top, bottom insights Added tests for top, bottom insights * max and min insightsperslice, and tests * got rid of unneccesssary function * get indexes * learn for stringinputtyype * add learn implentation * add unique inputs * fix merge error * add to result Co-authored-by: Karl Burtram <karlb@microsoft.com> Co-authored-by: Aasim Khan <aasimkhan30@gmail.com> Co-authored-by: Arslan Memon <armemon@microsoft.com> * Added all the templates (#1015) * Added all the templates Added a method to find matched template * Added a function to replace # and ## values in a template * Added ReplaceHashesInTemplate call * Added comments * Updated the template txt * Updated GetTopHeadersWithHash function to add #toplist * Updated the logic per offline discussion with Hermineh * Update request handler contract to take array (#1020) * added rulesengine findmatchingtemplate (#1019) * Add support for getting DacFx deploy options from a publish profile (#995) * add support for getting options from a publish profile * update comments * set values for default options if they aren't specified in the publish profile * addressing comments * Updating to latest DacFx for a bug fix (#1010) * added rulesengine findmatchingtemplate * Update DacFx deploy and generate script with options (#998) * update deploy and generate script to accept deployment options * add tests * add test with option set to true * merge * merge * incorporated FindMatchedTemplate Co-authored-by: Kim Santiago <31145923+kisantia@users.noreply.github.com> Co-authored-by: Udeesha Gautam <46980425+udeeshagautam@users.noreply.github.com> * -Added logic for Insights Generator Service Handler (#1017) * -Added logic for Insights Generator Service Handler * Fixed some logic * Adding workflow test * Update transform and add tests (#1024) * Jiarima/fix rules engine logic (#1021) * Added all the templates Added a method to find matched template * Added a function to replace # and ## values in a template * Added ReplaceHashesInTemplate call * Added comments * Updated the template txt * Updated GetTopHeadersWithHash function to add #toplist * Updated the logic per offline discussion with Hermineh * Update with the fixes * Updated template and foreach conditions * Added distinct * Updated tests according to the logic change (#1026) * Nara/remove queing (#1023) * loc update (#914) * loc update * loc updates * Add support for getting DacFx deploy options from a publish profile (#995) * add support for getting options from a publish profile * update comments * set values for default options if they aren't specified in the publish profile * addressing comments * Updating to latest DacFx for a bug fix (#1010) * Update DacFx deploy and generate script with options (#998) * update deploy and generate script to accept deployment options * add tests * add test with option set to true * intermediate check in for merge, transformed not working * intermediate check in for merge, transformed not working * added test case * merged Co-authored-by: khoiph1 <khoiph@microsoft.com> Co-authored-by: Kim Santiago <31145923+kisantia@users.noreply.github.com> Co-authored-by: Udeesha Gautam <46980425+udeeshagautam@users.noreply.github.com> * Output data types from transform (#1029) * Fix bug process input_g (#1030) * Fixed the insight generator service (#1028) * Jiarima/added more testings (#1031) * Added another test Updated ReplaceHashesInTemplate function to return string instead of Template * Added third test * Merged * Reverted the workflow file to match with the one in hack/insights * Bugs fixes to hook insights up to ADS (#1033) * Bug fixes for hack insights (#1032) * Fixed the minColumn index bug in Data Transformation Fixed the template matching logic. * Adding changes from PR * Try to fix Workflow tests * Readd workflow tests * Fix template load location Co-authored-by: Aasim Khan <aasimkhan30@gmail.com> Co-authored-by: Nara <NaraVen@users.noreply.github.com> Co-authored-by: arslan9955 <53170027+arslan9955@users.noreply.github.com> Co-authored-by: Arslan Memon <armemon@microsoft.com> Co-authored-by: gadudhbh <68879970+gadudhbh@users.noreply.github.com> Co-authored-by: Jinjing Arima <jiarima@microsoft.com> Co-authored-by: jiarima <68882862+jiarima@users.noreply.github.com> Co-authored-by: Kim Santiago <31145923+kisantia@users.noreply.github.com> Co-authored-by: Udeesha Gautam <46980425+udeeshagautam@users.noreply.github.com> Co-authored-by: khoiph1 <khoiph@microsoft.com>
2026-02-08 09:38:25 -05:00 · 2020-09-03 14:15:51 -07:00
parent 9784c3eaa2
commit 5cf5b59a0d
29 changed files with 2281 additions and 12 deletions
--- a/test/Microsoft.InsightsGenerator.UnitTests/DataTransformationTests.cs
+++ b/test/Microsoft.InsightsGenerator.UnitTests/DataTransformationTests.cs
@@ -0,0 +1,198 @@
+//
+// Copyright (c) Microsoft. All rights reserved.
+// Licensed under the MIT license. See LICENSE file in the project root for full license information.
+//
+
+using Xunit;
+
+namespace Microsoft.InsightsGenerator.UnitTests
+{
+    /// <summary>
+    /// DataTransformation tests
+    /// </summary>
+    public class DataTransformerTests
+    {
+        [Fact]
+        public void Tranform_NullInput()
+        {
+            DataTransformer transformer = new DataTransformer();
+            DataArray array = null;
+            array = transformer.Transform(array);
+            Assert.Null(array);
+        }
+
+        [Fact]
+        public void Tranform_TimeSlicerCount_DeduceTypes()
+        {
+            DataTransformer transformer = new DataTransformer();
+            object[][] cells = new object[2][];
+            cells[0] = new object[3] { "1/15/2020", "Redmond", 50 };
+            cells[1] = new object[3] { "1/25/2020", "Bellevue", 75 };
+
+            DataArray array = new DataArray()
+            {
+                ColumnNames = new string[] { "Date", "City", "Count" },
+                Cells = cells
+            };
+
+            array = transformer.Transform(array);
+            Assert.Equal(array.TransformedColumnNames[0], "input_t_0");
+            Assert.Equal(array.TransformedColumnNames[1], "slicer_0");
+            Assert.Equal(array.TransformedColumnNames[2], "output_0");
+        }
+
+        [Fact]
+        public void Tranform_TimeSlicerCount_ProvidedTypes()
+        {
+            DataTransformer transformer = new DataTransformer();
+            object[][] cells = new object[2][];
+            cells[0] = new object[3] { "1/15/2020", "Redmond", 50 };
+            cells[1] = new object[3] { "1/25/2020", "Bellevue", 75 };
+
+            DataArray array = new DataArray()
+            {
+                ColumnNames = new string[] { "Date", "City", "Count" },
+                ColumnDataType = new DataArray.DataType[] {
+                    DataArray.DataType.String, 
+                    DataArray.DataType.DateTime, 
+                    DataArray.DataType.Number },
+                Cells = cells
+            };
+
+            array = transformer.Transform(array);
+            Assert.Equal(array.TransformedColumnNames[0], "slicer_0");
+            Assert.Equal(array.TransformedColumnNames[1], "input_t_0");
+            Assert.Equal(array.TransformedColumnNames[2], "output_0");
+        }
+
+        [Fact]
+        public void Tranform_TimeSlicerSlicerCount()
+        {
+            DataTransformer transformer = new DataTransformer();
+            object[][] cells = new object[5][];
+            cells[0] = new object[4] { "1/15/2020", "Redmond", "1st Street", 50 };
+            cells[1] = new object[4] { "1/25/2020", "Redmond", "2nd Street", 75 };
+            cells[2] = new object[4] { "1/10/2020", "Bellevue", "3rd Street", 125 };
+            cells[3] = new object[4] { "1/13/2020", "Bellevue", "4th Street", 55 };
+            cells[4] = new object[4] { "1/20/2020", "Bellevue", "5th Street", 95 };
+
+            DataArray array = new DataArray()
+            {
+                ColumnNames = new string[] { "Date", "City", "Address", "Count" },
+                Cells = cells
+            };
+
+            array = transformer.Transform(array);
+            Assert.Equal(array.TransformedColumnNames[0], "input_t_0");
+            Assert.Equal(array.TransformedColumnNames[1], "slicer_0");
+            Assert.Equal(array.TransformedColumnNames[2], "slicer_1");
+            Assert.Equal(array.TransformedColumnNames[3], "output_0");
+        }
+
+        [Fact]
+        public void Tranform_TimeSlicerCountSlicer()
+        {
+            DataTransformer transformer = new DataTransformer();
+            object[][] cells = new object[5][];
+            cells[0] = new object[4] { "1/15/2020", "1st Street", 50, "Redmond" };
+            cells[1] = new object[4] { "1/25/2020", "2nd Street", 75, "Redmond" };
+            cells[2] = new object[4] { "1/10/2020", "3rd Street", 125, "Bellevue" };
+            cells[3] = new object[4] { "1/13/2020", "4th Street", 55, "Bellevue" };
+            cells[4] = new object[4] { "1/20/2020", "5th Street", 95, "Bellevue" };
+
+            DataArray array = new DataArray()
+            {
+                ColumnNames = new string[] { "Date", "Address", "Count", "City" },
+                Cells = cells
+            };
+
+            array = transformer.Transform(array);
+            Assert.Equal(array.TransformedColumnNames[0], "input_t_0");
+            Assert.Equal(array.TransformedColumnNames[1], "slicer_0");
+            Assert.Equal(array.TransformedColumnNames[2], "output_0");
+            Assert.Equal(array.TransformedColumnNames[3], "slicer_1");
+        }
+
+        [Fact]
+        public void Tranform_TimeSlicerCountCount()
+        {
+            DataTransformer transformer = new DataTransformer();
+            object[][] cells = new object[2][];
+            cells[0] = new object[4] { "1/15/2020", "1st Street", 50, 110 };
+            cells[1] = new object[4] { "1/25/2020", "2nd Street", 75, 160 };
+
+            DataArray array = new DataArray()
+            {
+                ColumnNames = new string[] { "Date", "Adress", "Count1", "Count2" },
+                Cells = cells
+            };
+
+            array = transformer.Transform(array);
+            Assert.Equal(array.TransformedColumnNames[0], "input_t_0");
+            Assert.Equal(array.TransformedColumnNames[1], "slicer_0");
+            Assert.Equal(array.TransformedColumnNames[2], "output_0");
+            Assert.Equal(array.TransformedColumnNames[3], "output_1");
+        }
+
+        [Fact]
+        public void Tranform_GroupSlicerTime()
+        {
+            DataTransformer transformer = new DataTransformer();
+            object[][] cells = new object[2][];
+            cells[0] = new object[3] { "1st Street", "Redmond", 110 };
+            cells[1] = new object[3] { "2nd Street", "Bellevue", 160 };
+
+            DataArray array = new DataArray()
+            {
+                ColumnNames = new string[] { "Address", "City", "Count" },
+                Cells = cells
+            };
+
+            array = transformer.Transform(array);
+            Assert.Equal(array.TransformedColumnNames[0], "input_g_0");
+            Assert.Equal(array.TransformedColumnNames[1], "slicer_0");
+            Assert.Equal(array.TransformedColumnNames[2], "output_0");
+        }
+
+        [Fact]
+        public void Tranform_SlicewrGroupSlicerTime()
+        {
+            DataTransformer transformer = new DataTransformer();
+            object[][] cells = new object[2][];
+            cells[0] = new object[4] { "1st Street", "Redmond", "North", 110 };
+            cells[1] = new object[4] { "2nd Street", "Redmond", "East", 160 };
+
+            DataArray array = new DataArray()
+            {
+                ColumnNames = new string[] { "Address", "City", "Direction", "Count" },
+                Cells = cells
+            };
+
+            array = transformer.Transform(array);
+            Assert.Equal(array.TransformedColumnNames[0], "slicer_0");
+            Assert.Equal(array.TransformedColumnNames[1], "input_g_0");
+            Assert.Equal(array.TransformedColumnNames[2], "slicer_1");
+            Assert.Equal(array.TransformedColumnNames[3], "output_0");
+        }
+
+         [Fact]
+        public void Tranform_SlicerGroupTime()
+        {
+            DataTransformer transformer = new DataTransformer();
+            object[][] cells = new object[2][];
+            cells[0] = new object[3] { "1st Street", "Redmond", 110 };
+            cells[1] = new object[3] { "2nd Street", "Redmond", 160 };
+
+            DataArray array = new DataArray()
+            {
+                ColumnNames = new string[] { "Address", "City", "Count" },
+                Cells = cells
+            };
+
+            array = transformer.Transform(array);
+            Assert.Equal(array.TransformedColumnNames[0], "slicer_0");
+            Assert.Equal(array.TransformedColumnNames[1], "input_g_0");
+            Assert.Equal(array.TransformedColumnNames[2], "output_0");
+        }
+    }
+}
--- a/test/Microsoft.InsightsGenerator.UnitTests/Microsoft.InsightsGenerator.UnitTests.csproj
+++ b/test/Microsoft.InsightsGenerator.UnitTests/Microsoft.InsightsGenerator.UnitTests.csproj
@@ -0,0 +1,26 @@
+<Project Sdk="Microsoft.NET.Sdk">
+	<PropertyGroup Label="Configuration">
+		<OutputType>Exe</OutputType>
+		<TargetFramework>$(TestProjectsTargetFramework)</TargetFramework>
+		<GenerateAssemblyInfo>false</GenerateAssemblyInfo>
+		<DefineConstants>$(DefineConstants);TRACE</DefineConstants>
+		<IsPackable>false</IsPackable>
+		<ApplicationIcon />
+		<StartupObject />
+	</PropertyGroup>
+	<ItemGroup>
+		<PackageReference Include="System.Text.Encoding.CodePages" />
+		<PackageReference Include="Microsoft.NET.Test.Sdk" />
+		<PackageReference Include="Moq" />
+		<PackageReference Include="NUnit" />
+		<PackageReference Include="xunit" />
+		<PackageReference Include="xunit.runner.visualstudio" />
+		<PackageReference Include="Microsoft.Data.SqlClient" />
+	</ItemGroup>
+	<ItemGroup>
+		<ProjectReference Include="../../src/Microsoft.InsightsGenerator/Microsoft.InsightsGenerator.csproj" />
+	</ItemGroup>
+	<ItemGroup>
+		<Service Include="{82a7f48d-3b50-4b1e-b82e-3ada8210c358}" />
+	</ItemGroup>
+</Project>
--- a/test/Microsoft.InsightsGenerator.UnitTests/Properties/AssemblyInfo.cs
+++ b/test/Microsoft.InsightsGenerator.UnitTests/Properties/AssemblyInfo.cs
@@ -0,0 +1,41 @@
+//
+// Copyright (c) Microsoft. All rights reserved.
+// Licensed under the MIT license. See LICENSE file in the project root for full license information.
+//
+
+using System.Reflection;
+using System.Runtime.InteropServices;
+
+// General Information about an assembly is controlled through the following 
+// set of attributes. Change these attribute values to modify the information
+// associated with an assembly.
+[assembly: AssemblyTitle("Microsoft InsightsGenerator tests")]
+[assembly: AssemblyDescription("")]
+[assembly: AssemblyConfiguration("")]
+[assembly: AssemblyCompany("")]
+[assembly: AssemblyProduct("Microsoft InsightsGenerator tests")]
+[assembly: AssemblyCopyright("Copyright <20>  2020")]
+[assembly: AssemblyTrademark("")]
+[assembly: AssemblyCulture("")]
+
+// Setting ComVisible to false makes the types in this assembly not visible 
+// to COM components.  If you need to access a type in this assembly from 
+// COM, set the ComVisible attribute to true on that type.
+[assembly: ComVisible(false)]
+
+// The following GUID is for the ID of the typelib if this project is exposed to COM
+[assembly: Guid("68aa66d3-4d62-4ecf-85a0-3944256cb161")]
+
+// Version information for an assembly consists of the following four values:
+//
+//      Major Version
+//      Minor Version 
+//      Build Number
+//      Revision
+//
+// You can specify all the values or you can default the Build and Revision Numbers 
+// by using the '*' as shown below:
+// [assembly: AssemblyVersion("1.0.*")]
+[assembly: AssemblyVersion("1.0.0.0")]
+[assembly: AssemblyFileVersion("1.0.0.0")]
+
--- a/test/Microsoft.InsightsGenerator.UnitTests/RulesEngineTests.cs
+++ b/test/Microsoft.InsightsGenerator.UnitTests/RulesEngineTests.cs
@@ -0,0 +1,83 @@
+//
+// Copyright (c) Microsoft. All rights reserved.
+// Licensed under the MIT license. See LICENSE file in the project root for full license information.
+//
+
+using System;
+using System.Collections.Generic;
+using System.Linq;
+using Xunit;
+using static Microsoft.InsightsGenerator.RulesEngine;
+
+namespace Microsoft.InsightsGenerator.UnitTests
+{
+    /// <summary>
+    /// Rules Engine tests
+    /// </summary>
+    public class RulesEngineTests
+    {
+        [Fact]
+        public void TemplateParserTest()
+        {
+            ColumnHeaders headerForTemp8 = TemplateParser(@"#inp had a total of #OutPar_N_C_1 ##OutPar_N_C_1 that constitues #Per% \n \n");
+            ColumnHeaders headerForTemp16 = TemplateParser(@"For the #slices ##SlicePar_GG_1(s), the percentage of ##OutPar_N_C_1 on #time were \n #stHData\n this was compared with #Etime where #ESlices ##SlicePar_GG_1\n #EstHData  \n.");
+
+            var expectedSingleHashValuesForTemp16 = new List<string>(new string[] { "#slices", "#time", "#stHData", "#Etime", "#ESlices", "#EstHData" });
+            var expectedDoubleHashValuesForTemp16 = new List<string>(new string[] { "##SlicePar_GG_1(s)", "##OutPar_N_C_1", "##SlicePar_GG_1" });
+
+            var expectedSingleHashValuesForTemp8 = new List<string>(new string[] { "#inp", "#OutPar_N_C_1", "#Per%" });
+            var expectedDoubleHashValuesForTemp8 = new List<string>(new string[] { "##OutPar_N_C_1" });
+
+            Assert.True(Enumerable.SequenceEqual(expectedSingleHashValuesForTemp8, headerForTemp8.SingleHashValues));
+            Assert.True(Enumerable.SequenceEqual(expectedDoubleHashValuesForTemp8, headerForTemp8.DoubleHashValues));
+            Assert.True(Enumerable.SequenceEqual(expectedSingleHashValuesForTemp16, headerForTemp16.SingleHashValues));
+            Assert.True(Enumerable.SequenceEqual(expectedDoubleHashValuesForTemp16, headerForTemp16.DoubleHashValues));
+        }
+        [Fact]
+        public void RulesEngineEndToEndTest()
+        {
+            // Create test input objects for test #1
+            var singleHashList1 = new List<List<string>>();
+            var list1_1 = new List<string>() { "uniqueinputs", "15" };
+            var list1_2 = new List<string>() { "top", "3", "China: 55%", "United States: 49%", "Japan: 37%" };
+            singleHashList1.Add(list1_1);
+            singleHashList1.Add(list1_2);
+
+            DataArray testArray1 = new DataArray();
+            testArray1.ColumnNames = new string[] { "Country", "Area" };
+            testArray1.TransformedColumnNames = new string[] { "input_g_0", "output_0" };
+
+            // Create test input objects for test #2
+            var singleHashList2 = new List<List<string>>();
+            var list2_1 = new List<string>() { "bottom", "5", "Apple: 30%", "Oragne: 28%", "Strawberry: 17%", "Pear: 13%", "Peach: 8%" };
+            singleHashList2.Add(list2_1);
+
+            DataArray testArray2 = new DataArray();
+            testArray2.ColumnNames = new string[] { "fruits" };
+            testArray2.TransformedColumnNames = new string[] { "output_0" };
+
+            // Create test input objects for test#3
+            var singleHashList3 = new List<List<string>>();
+            var list3_1 = new List<string>() { "averageSlice", "4", "Cow: 60%", "Dog: 28%", "Cat: 17%", "Mouse: 8%"};
+            singleHashList3.Add(list3_1);
+
+            DataArray testArray3 = new DataArray();
+            testArray3.ColumnNames = new string[] { "animals" };
+            testArray3.TransformedColumnNames = new string[] { "slicer_0" };
+
+            var returnedStr1 = $@"{RulesEngine.FindMatchedTemplate(singleHashList1, testArray1)}";
+            var returnedStr2 = $@"{RulesEngine.FindMatchedTemplate(singleHashList2, testArray2)}";
+            var returnedStr3 = $@"{RulesEngine.FindMatchedTemplate(singleHashList3, testArray3)}";
+
+
+            string expectedOutput1 = "There were 15 Country (s),  the top 3 highest total Area were as follows:\\n China: 55%" + Environment.NewLine + "United States: 49%" + Environment.NewLine + "Japan: 37%" + Environment.NewLine + Environment.NewLine + Environment.NewLine;
+            string expectedOutput2 = "The top 5 lowest total fruits were as follows:\\n Apple: 30%" + Environment.NewLine + "Oragne: 28%" + Environment.NewLine + "Strawberry: 17%" + Environment.NewLine + "Pear: 13%" + Environment.NewLine + "Peach: 8%" + Environment.NewLine + Environment.NewLine + Environment.NewLine;
+            string expectedOutput3 = "For the 4 animals, the volume of each is: Cow: 60%" + Environment.NewLine + "Dog: 28%" + Environment.NewLine + "Cat: 17%" + Environment.NewLine + "Mouse: 8%" + Environment.NewLine + Environment.NewLine + Environment.NewLine;
+
+            Assert.True(string.Equals(returnedStr1, returnedStr1));
+            Assert.True(string.Equals(returnedStr2, expectedOutput2));
+            Assert.True(string.Equals(returnedStr3, expectedOutput3));
+
+        }
+    }
+}
--- a/test/Microsoft.InsightsGenerator.UnitTests/SignatureGeneratorTests.cs
+++ b/test/Microsoft.InsightsGenerator.UnitTests/SignatureGeneratorTests.cs
@@ -0,0 +1,340 @@
+using System;
+using System.Collections.Generic;
+using System.Text;
+using Xunit;
+
+
+namespace Microsoft.InsightsGenerator.UnitTests
+{
+    public class SignatureGeneratorTests
+    {
+
+        [Fact]
+        public void TopTest()
+        {
+            var expectedTopInsight = @"top
+3
+China (455) 19.73%
+Turkey (254) 11.01%
+United States (188) 8.15%";
+            SignatureGenerator sigGen = new SignatureGenerator(sampleDataArray(false));
+            sigGen.OverallTopInsights(3, 0, 1);
+
+            CompareInsightWithExpectedOutcome(sigGen.Result.Insights, expectedTopInsight);
+        }
+
+        [Fact]
+        public void TopSliceTest()
+        {
+            var expectedTopSliceInsight = @"topPerSlice
+5
+Category1
+3
+China (455) 34.89%
+Turkey (254) 19.48%
+United States (188) 14.42%
+Category2
+3
+Japan (171) 91.94%
+China (10) 5.38%
+United States (3) 1.61%
+Category3
+3
+United States (106) 15.5%
+Brazil (91) 13.3%
+Korea (61) 8.92%
+Category4
+3
+United States (38) 38%
+China (12) 12%
+Korea (8) 8%
+Category5
+3
+Korea (21) 65.62%
+United States (6) 18.75%
+Canada (3) 9.38%";
+            SignatureGenerator sigGen = new SignatureGenerator(sampleDataArray(false));
+            sigGen.SlicedTopInsights(3, 0, 2, 1);
+            CompareInsightWithExpectedOutcome(sigGen.Result.Insights, expectedTopSliceInsight);
+        }
+
+        [Fact]
+        public void BottomTest()
+        {
+            var expectedBottomInsight = @"bottom
+3
+Korea (1) 0.04%
+Germany (1) 0.04%
+India (1) 0.04%";
+            SignatureGenerator sigGen = new SignatureGenerator(sampleDataArray(false));
+            sigGen.OverallBottomInsights(3, 0, 1);
+            CompareInsightWithExpectedOutcome(sigGen.Result.Insights, expectedBottomInsight);
+        }
+
+        [Fact]
+        public void BottomSliceTest()
+        {
+            var expectedBottomSliceInsight = @"bottomPerSlice
+5
+Category1
+3
+Canada (17) 1.3%
+United Kingdom (17) 1.3%
+Vietnam (18) 1.38%
+Category2
+3
+Germany (1) 0.54%
+Korea (1) 0.54%
+United States (3) 1.61%
+Category3
+3
+France (12) 1.75%
+United Kingdom (20) 2.92%
+Vietnam (22) 3.22%
+Category4
+3
+India (1) 1%
+Japan (1) 1%
+Canada (2) 2%
+Category5
+3
+India (1) 3.12%
+Japan (1) 3.12%
+Canada (3) 9.38%";
+            SignatureGenerator sigGen = new SignatureGenerator(sampleDataArray(false));
+            sigGen.SlicedBottomInsights(3, 0, 2, 1);
+            CompareInsightWithExpectedOutcome(sigGen.Result.Insights, expectedBottomSliceInsight);
+        }
+
+        [Fact]
+        public void AverageTest()
+        {
+            var expectedAverageInsight = @"average
+42.7";
+            SignatureGenerator sigGen = new SignatureGenerator(sampleDataArray(false));
+            sigGen.OverallAverageInsights(1);
+            CompareInsightWithExpectedOutcome(sigGen.Result.Insights, expectedAverageInsight);
+        }
+
+        [Fact]
+        public void SumTest()
+        {
+            var expectedSumInsight = @"sum
+2306";
+            SignatureGenerator sigGen = new SignatureGenerator(sampleDataArray(false));
+            sigGen.OverallSumInsights(1);
+            CompareInsightWithExpectedOutcome(sigGen.Result.Insights, expectedSumInsight);
+        }
+
+        [Fact]
+        public void SlicedSumTest()
+        {
+            var expectedSlicedSumInsight = @"sumPerSlice
+5
+Category1
+1304
+Category2
+186
+Category3
+684
+Category4
+100
+Category5
+32";
+            SignatureGenerator sigGen = new SignatureGenerator(sampleDataArray(false));
+            sigGen.SlicedSumInsights(2, 1);
+            CompareInsightWithExpectedOutcome(sigGen.Result.Insights, expectedSlicedSumInsight);
+        }
+
+        [Fact]
+        public void SlicedAverageTest()
+        {
+            var expectedSlicedAverageInsight = @"sumPerSlice
+5
+Category1
+86.93
+Category2
+37.2
+Category3
+45.6
+Category4
+7.14
+Category5
+6.4";
+            SignatureGenerator sigGen = new SignatureGenerator(sampleDataArray(false));
+            sigGen.SlicedAverageInsights(2, 1);
+            CompareInsightWithExpectedOutcome(sigGen.Result.Insights, expectedSlicedAverageInsight);
+        }
+
+        [Fact]
+        public void SlicedPercentageTest()
+        {
+            var expectedSlicedPercentageInsight = @"percentagePerSlice
+5
+Category1
+56.55
+Category2
+8.07
+Category3
+29.66
+Category4
+4.34
+Category5
+1.39";
+            SignatureGenerator sigGen = new SignatureGenerator(sampleDataArray(false));
+            sigGen.SlicedPercentageInsights(2, 1);
+            CompareInsightWithExpectedOutcome(sigGen.Result.Insights, expectedSlicedPercentageInsight);
+        }
+
+        [Fact]
+        public void MaxAndMinInsightsTest()
+        {
+            var expectedMaxAndMinInsight = @"max
+455
+min
+1";
+            SignatureGenerator sigGen = new SignatureGenerator(sampleDataArray(false));
+            sigGen.OverallMaxInsights(1);
+            sigGen.OverallMinInsights(1);
+            CompareInsightWithExpectedOutcome(sigGen.Result.Insights, expectedMaxAndMinInsight);
+        }
+
+        [Fact]
+        public void MaxAndMinSlicedInsightsTest()
+        {
+            string expectedMaxAndMinSlicedInsight = @"maxPerSlice
+5
+Category1
+455
+Category2
+171
+Category3
+106
+Category4
+38
+Category5
+21
+minPerSlice
+5
+Category1
+17
+Category2
+1
+Category3
+12
+Category4
+1
+Category5
+1";
+            SignatureGenerator sigGen = new SignatureGenerator(sampleDataArray(false));
+            sigGen.SlicedMaxInsights(2, 1);
+            sigGen.SlicedMinInsights(2, 1);
+            CompareInsightWithExpectedOutcome(sigGen.Result.Insights, expectedMaxAndMinSlicedInsight);
+        }
+
+
+        public void CompareInsightWithExpectedOutcome(List<List<string>> insights, string expectedOutcome)
+        {
+            List<string> stringedInsights = new List<string>();
+            foreach (List<string> insight in insights)
+            {
+                stringedInsights.Add(string.Join(Environment.NewLine, insight));
+            }
+            Assert.Equal(expectedOutcome, string.Join(Environment.NewLine, stringedInsights));
+        }
+
+        [Fact]
+        public void LearnTest()
+        {
+            SignatureGenerator sigGen = new SignatureGenerator(sampleDataArray(false));
+            sigGen.Learn();
+            foreach (List<string> list in sigGen.Result.Insights)
+            {
+                foreach (string str in list)
+                {
+                    Console.WriteLine(str);
+                }
+            }
+        }
+        public DataArray sampleDataArray(bool timeinput)
+        {
+            DataArray sample = new DataArray();
+
+            var inputDataType = DataArray.DataType.String;
+            if (timeinput)
+            {
+                inputDataType = DataArray.DataType.DateTime;
+            }
+
+            sample.ColumnNames = new string[] { "input_g_0", "output_0", "slicer_0" };
+            sample.ColumnDataType = new DataArray.DataType[] { inputDataType, DataArray.DataType.Number, DataArray.DataType.String };
+            string sampleTableString =
+                @"China	455	Category1
+Turkey	254	Category1
+United States	188	Category1
+Japan	171	Category2
+United States	106	Category3
+Brazil	91	Category3
+Thailand	67	Category1
+Korea	61	Category3
+Russia	61	Category1
+China	60	Category3
+Brazil	57	Category1
+Germany	51	Category3
+Turkey	49	Category3
+Russia	45	Category3
+Japan	44	Category3
+United States	38	Category4
+Thailand	37	Category3
+India	36	Category3
+Germany	35	Category1
+France	33	Category1
+India	31	Category1
+Japan	28	Category1
+Mexico	27	Category3
+Canada	23	Category3
+Mexico	22	Category1
+Vietnam	22	Category3
+Korea	21	Category1
+Korea	21	Category5
+United Kingdom	20	Category3
+Vietnam	18	Category1
+Canada	17	Category1
+United Kingdom	17	Category1
+China	12	Category4
+France	12	Category3
+China	10	Category2
+Korea	8	Category4
+Brazil	6	Category4
+Russia	6	Category4
+United States	6	Category5
+France	5	Category4
+Germany	5	Category4
+United Kingdom	5	Category4
+Thailand	4	Category4
+Turkey	4	Category4
+Canada	3	Category5
+Mexico	3	Category4
+United States	3	Category2
+Canada	2	Category4
+Germany	1	Category2
+India	1	Category4
+India	1	Category5
+Japan	1	Category4
+Japan	1	Category5
+Korea	1	Category2";
+
+            string[] sampleRows = sampleTableString.Split(Environment.NewLine);
+            List<string[]> sampleRowList = new List<string[]>();
+            foreach (var row in sampleRows)
+            {
+                sampleRowList.Add(row.Split("	"));
+            }
+
+            var columnTypes = new string[] { "input_g_1", "output_1", "slicer_1" };
+            sample.Cells = sampleRowList.ToArray();
+            sample.TransformedColumnNames = columnTypes;
+            return sample;
+        }
+    }
+}
--- a/test/Microsoft.InsightsGenerator.UnitTests/WorkflowTests.cs
+++ b/test/Microsoft.InsightsGenerator.UnitTests/WorkflowTests.cs
@@ -0,0 +1,95 @@
+using System;
+using System.Collections.Generic;
+using System.Text;
+using Xunit;
+
+namespace Microsoft.InsightsGenerator.UnitTests
+{
+    public class WorkFlowTests
+    {
+        [Fact]
+        public async void mainWorkFlowTest()
+        {
+            Workflow instance = new Workflow();
+            string insight = await instance.ProcessInputData(getSampleDataArray());
+            Assert.NotNull(insight);
+        }
+
+        public DataArray getSampleDataArray()
+        {
+            string sampleTableString =
+                @"Country Count Category
+China	455	Category1
+Turkey	254	Category1
+United States	188	Category1
+Japan	171	Category2
+United States	106	Category3
+Brazil	91	Category3
+Thailand	67	Category1
+Korea	61	Category3
+Russia	61	Category1
+China	60	Category3
+Brazil	57	Category1
+Germany	51	Category3
+Turkey	49	Category3
+Russia	45	Category3
+Japan	44	Category3
+United States	38	Category4
+Thailand	37	Category3
+India	36	Category3
+Germany	35	Category1
+France	33	Category1
+India	31	Category1
+Japan	28	Category1
+Mexico	27	Category3
+Canada	23	Category3
+Mexico	22	Category1
+Vietnam	22	Category3
+Korea	21	Category1
+Korea	21	Category5
+United Kingdom	20	Category3
+Vietnam	18	Category1
+Canada	17	Category1
+United Kingdom	17	Category1
+China	12	Category4
+France	12	Category3
+China	10	Category2
+Korea	8	Category4
+Brazil	6	Category4
+Russia	6	Category4
+United States	6	Category5
+France	5	Category4
+Germany	5	Category4
+United Kingdom	5	Category4
+Thailand	4	Category4
+Turkey	4	Category4
+Canada	3	Category5
+Mexico	3	Category4
+United States	3	Category2
+Canada	2	Category4
+Germany	1	Category2
+India	1	Category4
+India	1	Category5
+Japan	1	Category4
+Japan	1	Category5
+Korea	1	Category2";
+
+            string[] sampleRows = sampleTableString.Split(Environment.NewLine);
+
+
+            var columnNames = sampleRows[0].Split(" ");
+
+            List<string[]> sampleRowList = new List<string[]>();
+
+            for (int i = 1; i < sampleRows.Length; i++)
+            {
+                sampleRowList.Add(sampleRows[i].Split("	"));
+            }
+
+            DataArray result = new DataArray();
+            result.ColumnNames = columnNames;
+            result.Cells = sampleRowList.ToArray();
+            return result;
+        }
+    }
+}