* 3647 Renamed ParentMetadata to DatabaseMetadata in FolderMetadata.cs. Removed Refresh function from IDataSource and DataSourceBase since it's not used outside of KustoDataSource. Added unit tests for ServerNode.cs. Removed unused constructor from TreeNode and unused AddChild function.
* 3647 Refactored KustoDataSource. Moved Set logic for DatabaseMetadata and columns into separate functions
* 3647 Reverted Rename of ParentMetadata in FolderMetadata. Reverted removal of Refresh function from DataSourceBase and IDatasource
* 3647 Renamed SmoModel directories to DataSourceModel in Kusto projects.
* 3647 Removed reference to Kusto.ServiceLayer in SqlTools.ServiceLayer
* InsightsGenerator project template files
* Add insights projects to SLN
* Setting up siggen class (#1003)
* Setting up siggen class
* fixed the learn method
* Refactoring code
Fixed compile errors
* renamed results to result
* Basic transformation logic (#1004)
* Fix a couple bugs and add a simple test (#1007)
* Fix a couple bugs and add a simple test
* More tests and bug fix
* Nara/workflow (#1006)
* added a queue processor
* ordered using statements
* Armemon/analytics (#1008)
* Basic transformation logic
* changed some structure of siggen
* added sum and average method, as well as select rows by input name
* add insights to results
* min, max added
Co-authored-by: Karl Burtram <karlb@microsoft.com>
Co-authored-by: Aasim Khan <aasimkhan30@gmail.com>
Co-authored-by: Arslan Memon <armemon@microsoft.com>
* Added rules engine base implementation (#1005)
* Added rules engine base implementation
* update comments
* addressing comments
* adding template text to columnheaders object
* adding template text to columnheaders object
* fixing columnheaders class
* Added test
* Added Template Parser unit test in Test project
* Deleted unnecessary files and reverted the files that were modified by mistake
Co-authored-by: Jinjing Arima <jiarima@microsoft.com>
* Insights generator message handler placeholder (#1013)
* Aasim/insights/insight methods (#1014)
* Basic transformation logic
* changed some structure of siggen
* Added top and bottom insight functions
* Added top, bottom insights
Added tests for top, bottom insights
* Armemon/insights2 (#1011)
* max and min insightsperslice, and tests
* got rid of unneccesssary function
* get indexes
Co-authored-by: Arslan Memon <armemon@microsoft.com>
* Armemon/insights2 (#1012)
* max and min insightsperslice, and tests
* got rid of unneccesssary function
* get indexes
* learn for stringinputtyype
* add learn implentation
Co-authored-by: Arslan Memon <armemon@microsoft.com>
* Added Tests
Removed duplicate methods
Co-authored-by: Karl Burtram <karlb@microsoft.com>
Co-authored-by: arslan9955 <53170027+arslan9955@users.noreply.github.com>
Co-authored-by: Arslan Memon <armemon@microsoft.com>
* Armemon/insights2 (#1016)
* Basic transformation logic
* changed some structure of siggen
* Added top and bottom insight functions
* Added top, bottom insights
Added tests for top, bottom insights
* max and min insightsperslice, and tests
* got rid of unneccesssary function
* get indexes
* learn for stringinputtyype
* add learn implentation
* add unique inputs
* fix merge error
* add to result
Co-authored-by: Karl Burtram <karlb@microsoft.com>
Co-authored-by: Aasim Khan <aasimkhan30@gmail.com>
Co-authored-by: Arslan Memon <armemon@microsoft.com>
* Added all the templates (#1015)
* Added all the templates
Added a method to find matched template
* Added a function to replace # and ## values in a template
* Added ReplaceHashesInTemplate call
* Added comments
* Updated the template txt
* Updated GetTopHeadersWithHash function to add #toplist
* Updated the logic per offline discussion with Hermineh
* Update request handler contract to take array (#1020)
* added rulesengine findmatchingtemplate (#1019)
* Add support for getting DacFx deploy options from a publish profile (#995)
* add support for getting options from a publish profile
* update comments
* set values for default options if they aren't specified in the publish profile
* addressing comments
* Updating to latest DacFx for a bug fix (#1010)
* added rulesengine findmatchingtemplate
* Update DacFx deploy and generate script with options (#998)
* update deploy and generate script to accept deployment options
* add tests
* add test with option set to true
* merge
* merge
* incorporated FindMatchedTemplate
Co-authored-by: Kim Santiago <31145923+kisantia@users.noreply.github.com>
Co-authored-by: Udeesha Gautam <46980425+udeeshagautam@users.noreply.github.com>
* -Added logic for Insights Generator Service Handler (#1017)
* -Added logic for Insights Generator Service Handler
* Fixed some logic
* Adding workflow test
* Update transform and add tests (#1024)
* Jiarima/fix rules engine logic (#1021)
* Added all the templates
Added a method to find matched template
* Added a function to replace # and ## values in a template
* Added ReplaceHashesInTemplate call
* Added comments
* Updated the template txt
* Updated GetTopHeadersWithHash function to add #toplist
* Updated the logic per offline discussion with Hermineh
* Update with the fixes
* Updated template and foreach conditions
* Added distinct
* Updated tests according to the logic change (#1026)
* Nara/remove queing (#1023)
* loc update (#914)
* loc update
* loc updates
* Add support for getting DacFx deploy options from a publish profile (#995)
* add support for getting options from a publish profile
* update comments
* set values for default options if they aren't specified in the publish profile
* addressing comments
* Updating to latest DacFx for a bug fix (#1010)
* Update DacFx deploy and generate script with options (#998)
* update deploy and generate script to accept deployment options
* add tests
* add test with option set to true
* intermediate check in for merge, transformed not working
* intermediate check in for merge, transformed not working
* added test case
* merged
Co-authored-by: khoiph1 <khoiph@microsoft.com>
Co-authored-by: Kim Santiago <31145923+kisantia@users.noreply.github.com>
Co-authored-by: Udeesha Gautam <46980425+udeeshagautam@users.noreply.github.com>
* Output data types from transform (#1029)
* Fix bug process input_g (#1030)
* Fixed the insight generator service (#1028)
* Jiarima/added more testings (#1031)
* Added another test
Updated ReplaceHashesInTemplate function to return string instead of Template
* Added third test
* Merged
* Reverted the workflow file to match with the one in hack/insights
* Bugs fixes to hook insights up to ADS (#1033)
* Bug fixes for hack insights (#1032)
* Fixed the minColumn index bug in Data Transformation
Fixed the template matching logic.
* Adding changes from PR
* Try to fix Workflow tests
* Readd workflow tests
* Fix template load location
Co-authored-by: Aasim Khan <aasimkhan30@gmail.com>
Co-authored-by: Nara <NaraVen@users.noreply.github.com>
Co-authored-by: arslan9955 <53170027+arslan9955@users.noreply.github.com>
Co-authored-by: Arslan Memon <armemon@microsoft.com>
Co-authored-by: gadudhbh <68879970+gadudhbh@users.noreply.github.com>
Co-authored-by: Jinjing Arima <jiarima@microsoft.com>
Co-authored-by: jiarima <68882862+jiarima@users.noreply.github.com>
Co-authored-by: Kim Santiago <31145923+kisantia@users.noreply.github.com>
Co-authored-by: Udeesha Gautam <46980425+udeeshagautam@users.noreply.github.com>
Co-authored-by: khoiph1 <khoiph@microsoft.com>
* 3278 Added unit tests in MetadataFactoryTests and Microsoft.Kusto.ServiceLayer.UnitTests project
* 3278 Removed todo and changed unit test to validate megabytes
* 3278 Added file and unit tests in AutoCompleteHelperTests.cs
* 3278 Removed unused functions from Kusto > ScriptAsScriptingOperation
* 3278 Added unit tests for DataSourceFactory
* 3278 Refactored AdminService to pass in ConnectionService rather than through instance variable. Added unit test for AdminServiceTests
* 3278 Refactored DataSourceFactory to not have static functions for future unit tests
* 3278 Re-added properties that were flagged as unused but are being used by ADS in ReliableDataSourceConnection.cs
* 3278 Re-added properties that were flagged as unused but are being used by ADS in ReliableDataSourceConnection.cs
* adding pipeline to execute tests (#1062)
* 3278 Converted GetDefaultAutoComplete and GetDefaultSemanticMarkers to static functions in DataSourceFactory. Removed unused constructor in ScriptFile. Added positive unit tests for both functions
* undoing release version bump
* adding additional configs
* adressing feedback
* Correcting path in csproj
Co-authored-by: Jorge Berumen <52225468+joberume@users.noreply.github.com>
Co-authored-by: joberume <jberumen3@miners.utep.edu>
* Remove xunit dependency from testdriver
* swap expected/actual as needed
* Convert Test.Common to nunit
* port hosting unit tests to nunit
* port batchparser integration tests to nunit
* port testdriver.tests to nunit
* fix target to copy dependency
* port servicelayer unittests to nunit
* more unit test fixes
* port integration tests to nunit
* fix test method type
* try using latest windows build for PRs
* reduce test memory use
* Update form .NET Core 2.2 to .NET Core 3.1
- Global variable for projects
- Change TFMs from netcoreapp2.2 to netcoreapp3.1
- Update global.json
- Update build.json
- Remove direct framework cake TestCore task
- Update travis dotnet version
- Update azure pipline file
- Update vscode launch.json
* Add Central Package Management
* Fix xUnit Breaking Change for MemberData type
* Fix xUnit breaking change for duplicate test method name
* Fix Rang/Index type conflict with System.Rang/Index
* Update vscode tasks.json
* Change serviceHostExecutable path in ServiceTestDriver.cs
* Downgrade SDK version (https://github.com/appveyor/ci/issues/3440)
- Appveyor hasn't installed latest SDK therefore I downgrade it until they install it.
* Dump Microsoft.SqlServer.DACFx
* Revert "Update SMO (#875)"
This reverts commit 96593d66e3.
* Revert "Switch Tools Service to new SMO and Microsoft.Data.SqlClient driver (#865)"
This reverts commit 9d140b53f3.
* Bump SMO to 160.1910315.0-preview
* send back success of include/exclude request
* update dacfx nuget package and use new api to get affected dependencies of include/exclude request
* addressing comments
* rename test
* Addressing comments
* switch to ambient props and targets files
* build against Microsoft.Data.SqlClient
* build tests
* fix test bug
* temporarily add SMO nuget to the repo
* update to released Microsoft.Data package
* pass AzureAcountToken if there is one
* Addressing comments
* formatting
* more formatting and moving getting connectionstring from constructor to CreateSchemaCompareEndpoint()
* Added endpoint for fetching all notebook jobs
* Refractored NotebookJobInfo to AgentNotebookInfo to make it more consistent with the rest of the codebase
* Added Notebook History endpoint in contracts.
* Added Create, Update, Delete notebook endpoints. Also added separate fetch template, materialized notebook endpoints. This will make the Notebook Request and Notebook History responses lighter.
* AgentNotebookInfo is now derived from AgentJobInfo
* added fetch noteook history endpoint
* Added fetching materialized notebook endpoint
* Added code for cleaning up the directory
* Added create notebook api
* Added Update and delete notebook job
* Fixed notebook history API
* Added last run info to the script and template folder
* Added execute database feature for notebook Jobs
* SQL commands are now using sqlparameters to prevent
any injection attacks
* Changed rundate and runtime to string to preserve
leading zeros
* integration test for agentnotebooks api
* Made some changes mentioned in PR
* Refactored the code, removed enpoint logic from the notebook handler and
wrote test cases
* changes select statements, fixed a bug in the test job cleanup
and fixed other stuff mentioned in the PR.
* added notebook_error column in notebook history select statement
* Added get template notebook endpoint
* updating dacfx with private nuget for testing
* Updating to real dacfx-preview version
* Removing additional refs added by nuget package manager (these were not there previously)
* successfully sends list of differences to ADS
* bumping dacfx nuget package version
* add second schema compare try for case when first db to db comparison fails but works on second try
* move schemacompare out of dacfx folders
* capitalizing
* more capitalizing
* addressing comments
* Switch to the unified SMO NuGet so all binaries are strong named and signed.
* use one AssemblyLoader instance for all loads
* Revert "use one AssemblyLoader instance for all loads"
This reverts commit 48c59ffd5c57152de281c87acdbcad7ddf7ab760.
* Stop creating multiple AssemblyLoadContext objects during composition, per https://github.com/dotnet/coreclr/issues/19632
* restore high entropyva property
* Improve the comment
* Update SMO to 150 and fix scripting for objects with a single quote in their name.
* restore newtonsoft reference
This change modifies the logging framework within sqltoolservice.
Moves away from custom Logger object to start using .Net tracing framework. It supports for the static Trace and TraceSource way of logging. For all new code it is recommend that we log the log messages using the existing static Logger class, while the code changes will continue to route the older Trace.Write* calls from the process to same log listeners (and thus the log targets) as used by the Logger class. Thus tracing in SMO code that uses Trace.Write* methods gets routed to the same file as the messages from rest of SQLTools Service code.
Make changes to start using .Net Frameworks codebase for all logging to unify our logging story.
Allows parameter to set tracingLevel filters that controls what kinds of message make it to the log file.
Allows a parameter to set a specific log file name so if these gets set by external code (the UI code using the tools service for example) then the external code is aware of the current log file in use.
Adding unittests to test out the existing and improved logging capabilities.
Sequences of checkins in development branch:
* Saving v1 of logging to prepare for code review. Minor cleanup and some end to end testing still remains
* Removing local launchSettings.json files
* added support for lazy listener to sqltoolsloglistener and removed incorrect changes to comments across files in previous checkin
* Converting time to local time when writing entries to the log
* move the hosting.v2 to new .net based logging code
* removing *.dgml files and addding them to .gitignore
* fixing typo of defaultTraceSource
* Addressing pull request feedback
* Adding a test to verify logging from SMO codebase
* propogating changes to v1 sqltools.hosting commandoptions.cs to the v2 version
* Fixing comments on start and stop callstack methods and whitespaces
* Commenting a test that got uncommented by mistake
* addding .gitattributes file as .sql file was observed to be misconstrued as a binary file
* Revert "fixed perf tests and added more scenarios (#683)"
This reverts commit aa2b30f486.
* Revert "Switch to the unified SMO NuGet so all binaries are strong named and signed. (#682)"
This reverts commit 69961992bb.
* Upgrade SMO packages to fix issue with scripting tables when Script Dependencies is true on the script options.
* use linux style import with case matching file name
* Refactoring sql script formatting helpers into To and From helpers
* Updates to make error messages for formatting errors more useful
* Fixing dumb breaks in unit tests
* Addressing comments from PR
* Updates to the SR files...
