* The ServerType of SQL OD in SMO is being changed to cloud. This is a minimal set of changes needed in sqltoolservice in order to keep everything working within ADS.
* SMO version update
* Fix for queries with comments separated by new lines
* Mimic Kusto behaviour for comments ran as query
* extra check
* address comment
Co-authored-by: Monica Gupta <mogupt@microsoft.com>
* Revert "Revert "Initial migration service changes for assessment support (#1093)" (#1098)"
This reverts commit e96c0064ad.
* Move SQL Tools Service last in Main project list
* 3490 Injected OwnerUri into KustClient to store for token refreshing. Removed UpdateAzureToken from IDataSource, DataSourceBase, and KustoDataSource. Removed logic for retrying queries related to Unauthorized datasource in Batch and Query. Changed ScriptingService, ScriptingScriptOperation, and ScriptAsScriptingOperation to take DataSource in the constructor instead of datasourcefactory. Changed ScriptingService to inject ConnectionService through InitializeService function.
* 3490 Removed Catch block for DataSourceUnauthorizedException in ExecuteControlCommandAsync
* 3490 Removed OwnerUri from KustoClient and used azureAccountToken to refresh token in ConnectionService.
* 3490 Reverted unneeded changes.
* 3490 Split ExecuteQuery in KustoClient to execute first query then remaining queries after
* 3490 Passed OwnerUri down into KustoClient to refresh token.
* 3490 Removed DataSourceUnauthorizedException. Refactored ExecuteQuery to catch aggregate exception. Added RefreshAzureToken logic to ExecuteControlCommand
* 3490 Added logic to update ReliableDataSourceConnection azure token within ConnectionInfo.
* 3490 Add retry logic to ExecuteQuery and ExecuteControlCommand in KustoClient
* 3326 Removed sqlConnection.Open from CreateSqlConnection in ConnectionService.
* 3326 Removed unused variables and functions from ConnectedBindingContext, ConnectedBindingQueue, IBindingContext, and IConnectedBindingQueue
* 3326 Removed unused constant SqlAzureEdition and unused functions CreateSqlConnection and CreateServerConnection from ConnectionService. Removed ServerConnection, MetadataDisplayInfoProvider, Binder, and SmoMetaDataProvider from ConnectedBindingContext. Deleted SqlConnectionOpener and ISqlConnectionOpener.
* 3644 Created KustoClient to handle interaction with Kusto servers. Refactored KustoDataSource and moved execution logic into KustoClient.
* 3644 Added RequestSecurityTokenParams, RequestSecurityTokenResponse, and SecurityTokenRequest to Kusto. Moved intellisense functions from KustoDataSource to KustoIntellisenseHelper. Added SchemaState to readonly property on KustoDataSource. Added catch block to Batch.cs to catch KustoRequestExceptions.
* 3644 Removed unused reference from ConnectionDetails and ConnectedBindingContext. Added UpdateAzureToken function to IKustoClient, KustoClient, IDataSource, KustoDataSource, and KustoDataSource. Added dataSource.Dispose to ReliableDataSourceConnection > Close. Added RefreshAzureToken to ConnectionService to refresh azure token.
* 3644 Removed unused properties from RequestSecurityTokenParams and RequestSecurityTokenResponse
* 3644 Added default to DbColumnWrapper to UnknownTypeName when null. Moved database query logic from KustoIntellisenseHelper to KustoClient. Moved KustoIntellisenseHelper data objects out of class. Changed SchemaState to load through separate tasks.
* 3644 Changed ReRunQuery logic in Kusto Batch to flip back to false if the query fails a second time so it can be rejected
* 3644 Updated GetAutoCompleteSuggestions and GetHoverHelp with changes from main
* 3644 Added AccountId to RequestSecurityTokenParams and set value in ConnectionService. Added throw to Batch.cs to ensure exceptions that are non-Unauthorized bubble up to ADS.
* 3644 Changed KustoUnauthorizedException to take original exception as inner exception. Changed catch block to only throw KustoUnauthorizedException when FailureCode is 401
* 3644 Renamed KustoUnauthorizedException to DataSourceUnauthorizedException. Moved logic to throw exception down into KustoClient. Changed retryLogic in Batch.cs to a decrementing count
* 3644 Changed logic in Batch.cs for throwing InvalidOperationException
* Clearing first use of parameters
* restore space
* added finally block
* Added try catch block in correct position
* fixed spacing
* added space before try
* duplicate row check simplified by Alan
* Revert "duplicate row check simplified by Alan"
This reverts commit 445ac506bd96b353266778abd0cf9bad2be2a3c3.
* removed exception and changed message
* added exception back for test purposes
* added working executescalar
* Added "<=" for delete action that deletes nothing.
* spacing fix for check duplicate
* Added comments and changed function logic.
* added clarifying message
* Added new extended class
* small space fix
* 3715 Refactored GetDatabaseName to load database names from kusto server when no database name available in connection string.
* 3715 Removed variable name from GetDatabaseName
* 3643 Enhanced GetAutoCompleteSuggestions to escape spaces in the insertText when generating list of suggestions. Changed detail to include kind.
* 3643 Fixed formatting
* 3643 Changed variable in GetAutoCompleteSuggestions
* 3647 Renamed ParentMetadata to DatabaseMetadata in FolderMetadata.cs. Removed Refresh function from IDataSource and DataSourceBase since it's not used outside of KustoDataSource. Added unit tests for ServerNode.cs. Removed unused constructor from TreeNode and unused AddChild function.
* 3647 Refactored KustoDataSource. Moved Set logic for DatabaseMetadata and columns into separate functions
* 3647 Reverted Rename of ParentMetadata in FolderMetadata. Reverted removal of Refresh function from DataSourceBase and IDatasource
* 3647 Renamed SmoModel directories to DataSourceModel in Kusto projects.
* 3647 Removed reference to Kusto.ServiceLayer in SqlTools.ServiceLayer
* 3278 Added unit tests in MetadataFactoryTests and Microsoft.Kusto.ServiceLayer.UnitTests project
* 3278 Removed todo and changed unit test to validate megabytes
* 3278 Added file and unit tests in AutoCompleteHelperTests.cs
* 3278 Removed unused functions from Kusto > ScriptAsScriptingOperation
* 3278 Added unit tests for DataSourceFactory
* 3278 Refactored AdminService to pass in ConnectionService rather than through instance variable. Added unit test for AdminServiceTests
* 3278 Refactored DataSourceFactory to not have static functions for future unit tests
* 3278 Re-added properties that were flagged as unused but are being used by ADS in ReliableDataSourceConnection.cs
* 3278 Re-added properties that were flagged as unused but are being used by ADS in ReliableDataSourceConnection.cs
* adding pipeline to execute tests (#1062)
* 3278 Converted GetDefaultAutoComplete and GetDefaultSemanticMarkers to static functions in DataSourceFactory. Removed unused constructor in ScriptFile. Added positive unit tests for both functions
* undoing release version bump
* adding additional configs
* 3278 Minor refactors in ConnectionInfo, BindingQueue, DiagnosticsHelper, MetadataService, and HostLoader. Changed AssemblyInfo to only allow Kusto Unit Tests for internal access. Added lots of unit tests.
* 3278 Commented out bindingContext.IsConnected in AddConnectionContext_Sets_BindingContext
* 3278 Reversed order of unit tests in ConnectedBindingQueueTests and added throw to Catch block.
* 3278 Reverted change to ConnectedBindingQueue. Removed unit test from AddConnectionContext for NeedsMetaData True
Co-authored-by: Jorge Berumen <52225468+joberume@users.noreply.github.com>
Co-authored-by: joberume <jberumen3@miners.utep.edu>
* InsightsGenerator project template files
* Add insights projects to SLN
* Setting up siggen class (#1003)
* Setting up siggen class
* fixed the learn method
* Refactoring code
Fixed compile errors
* renamed results to result
* Basic transformation logic (#1004)
* Fix a couple bugs and add a simple test (#1007)
* Fix a couple bugs and add a simple test
* More tests and bug fix
* Nara/workflow (#1006)
* added a queue processor
* ordered using statements
* Armemon/analytics (#1008)
* Basic transformation logic
* changed some structure of siggen
* added sum and average method, as well as select rows by input name
* add insights to results
* min, max added
Co-authored-by: Karl Burtram <karlb@microsoft.com>
Co-authored-by: Aasim Khan <aasimkhan30@gmail.com>
Co-authored-by: Arslan Memon <armemon@microsoft.com>
* Added rules engine base implementation (#1005)
* Added rules engine base implementation
* update comments
* addressing comments
* adding template text to columnheaders object
* adding template text to columnheaders object
* fixing columnheaders class
* Added test
* Added Template Parser unit test in Test project
* Deleted unnecessary files and reverted the files that were modified by mistake
Co-authored-by: Jinjing Arima <jiarima@microsoft.com>
* Insights generator message handler placeholder (#1013)
* Aasim/insights/insight methods (#1014)
* Basic transformation logic
* changed some structure of siggen
* Added top and bottom insight functions
* Added top, bottom insights
Added tests for top, bottom insights
* Armemon/insights2 (#1011)
* max and min insightsperslice, and tests
* got rid of unneccesssary function
* get indexes
Co-authored-by: Arslan Memon <armemon@microsoft.com>
* Armemon/insights2 (#1012)
* max and min insightsperslice, and tests
* got rid of unneccesssary function
* get indexes
* learn for stringinputtyype
* add learn implentation
Co-authored-by: Arslan Memon <armemon@microsoft.com>
* Added Tests
Removed duplicate methods
Co-authored-by: Karl Burtram <karlb@microsoft.com>
Co-authored-by: arslan9955 <53170027+arslan9955@users.noreply.github.com>
Co-authored-by: Arslan Memon <armemon@microsoft.com>
* Armemon/insights2 (#1016)
* Basic transformation logic
* changed some structure of siggen
* Added top and bottom insight functions
* Added top, bottom insights
Added tests for top, bottom insights
* max and min insightsperslice, and tests
* got rid of unneccesssary function
* get indexes
* learn for stringinputtyype
* add learn implentation
* add unique inputs
* fix merge error
* add to result
Co-authored-by: Karl Burtram <karlb@microsoft.com>
Co-authored-by: Aasim Khan <aasimkhan30@gmail.com>
Co-authored-by: Arslan Memon <armemon@microsoft.com>
* Added all the templates (#1015)
* Added all the templates
Added a method to find matched template
* Added a function to replace # and ## values in a template
* Added ReplaceHashesInTemplate call
* Added comments
* Updated the template txt
* Updated GetTopHeadersWithHash function to add #toplist
* Updated the logic per offline discussion with Hermineh
* Update request handler contract to take array (#1020)
* added rulesengine findmatchingtemplate (#1019)
* Add support for getting DacFx deploy options from a publish profile (#995)
* add support for getting options from a publish profile
* update comments
* set values for default options if they aren't specified in the publish profile
* addressing comments
* Updating to latest DacFx for a bug fix (#1010)
* added rulesengine findmatchingtemplate
* Update DacFx deploy and generate script with options (#998)
* update deploy and generate script to accept deployment options
* add tests
* add test with option set to true
* merge
* merge
* incorporated FindMatchedTemplate
Co-authored-by: Kim Santiago <31145923+kisantia@users.noreply.github.com>
Co-authored-by: Udeesha Gautam <46980425+udeeshagautam@users.noreply.github.com>
* -Added logic for Insights Generator Service Handler (#1017)
* -Added logic for Insights Generator Service Handler
* Fixed some logic
* Adding workflow test
* Update transform and add tests (#1024)
* Jiarima/fix rules engine logic (#1021)
* Added all the templates
Added a method to find matched template
* Added a function to replace # and ## values in a template
* Added ReplaceHashesInTemplate call
* Added comments
* Updated the template txt
* Updated GetTopHeadersWithHash function to add #toplist
* Updated the logic per offline discussion with Hermineh
* Update with the fixes
* Updated template and foreach conditions
* Added distinct
* Updated tests according to the logic change (#1026)
* Nara/remove queing (#1023)
* loc update (#914)
* loc update
* loc updates
* Add support for getting DacFx deploy options from a publish profile (#995)
* add support for getting options from a publish profile
* update comments
* set values for default options if they aren't specified in the publish profile
* addressing comments
* Updating to latest DacFx for a bug fix (#1010)
* Update DacFx deploy and generate script with options (#998)
* update deploy and generate script to accept deployment options
* add tests
* add test with option set to true
* intermediate check in for merge, transformed not working
* intermediate check in for merge, transformed not working
* added test case
* merged
Co-authored-by: khoiph1 <khoiph@microsoft.com>
Co-authored-by: Kim Santiago <31145923+kisantia@users.noreply.github.com>
Co-authored-by: Udeesha Gautam <46980425+udeeshagautam@users.noreply.github.com>
* Output data types from transform (#1029)
* Fix bug process input_g (#1030)
* Fixed the insight generator service (#1028)
* Jiarima/added more testings (#1031)
* Added another test
Updated ReplaceHashesInTemplate function to return string instead of Template
* Added third test
* Merged
* Reverted the workflow file to match with the one in hack/insights
* Bugs fixes to hook insights up to ADS (#1033)
* Bug fixes for hack insights (#1032)
* Fixed the minColumn index bug in Data Transformation
Fixed the template matching logic.
* Adding changes from PR
* Try to fix Workflow tests
* Readd workflow tests
* Fix template load location
Co-authored-by: Aasim Khan <aasimkhan30@gmail.com>
Co-authored-by: Nara <NaraVen@users.noreply.github.com>
Co-authored-by: arslan9955 <53170027+arslan9955@users.noreply.github.com>
Co-authored-by: Arslan Memon <armemon@microsoft.com>
Co-authored-by: gadudhbh <68879970+gadudhbh@users.noreply.github.com>
Co-authored-by: Jinjing Arima <jiarima@microsoft.com>
Co-authored-by: jiarima <68882862+jiarima@users.noreply.github.com>
Co-authored-by: Kim Santiago <31145923+kisantia@users.noreply.github.com>
Co-authored-by: Udeesha Gautam <46980425+udeeshagautam@users.noreply.github.com>
Co-authored-by: khoiph1 <khoiph@microsoft.com>
* 3278 Added unit tests in MetadataFactoryTests and Microsoft.Kusto.ServiceLayer.UnitTests project
* 3278 Removed todo and changed unit test to validate megabytes
* 3278 Added file and unit tests in AutoCompleteHelperTests.cs
* 3278 Removed unused functions from Kusto > ScriptAsScriptingOperation
* 3278 Added unit tests for DataSourceFactory
* 3278 Refactored AdminService to pass in ConnectionService rather than through instance variable. Added unit test for AdminServiceTests
* 3278 Refactored DataSourceFactory to not have static functions for future unit tests
* 3278 Re-added properties that were flagged as unused but are being used by ADS in ReliableDataSourceConnection.cs
* 3278 Re-added properties that were flagged as unused but are being used by ADS in ReliableDataSourceConnection.cs
* adding pipeline to execute tests (#1062)
* 3278 Converted GetDefaultAutoComplete and GetDefaultSemanticMarkers to static functions in DataSourceFactory. Removed unused constructor in ScriptFile. Added positive unit tests for both functions
* undoing release version bump
* adding additional configs
* adressing feedback
* Correcting path in csproj
Co-authored-by: Jorge Berumen <52225468+joberume@users.noreply.github.com>
Co-authored-by: joberume <jberumen3@miners.utep.edu>
* INSERT VALUES has the limit of 1000 rows. Replace with derived table.
* Remove NOT NULL restriction from the generated table.
* Fix line endings in SQL Assessment source files.
* 3491 Added GenerateExecuteFunctionScript to IDataSource, DataSourceBase, and KustoDataSource. Added ExecutionFunction to Scripter and IScripter.
* 3491 Refactored GenerateScriptForFunction in ScriptAsScriptingOperation to handle execute and alter
* 3278 Moved functions related to Metadata from DataSourceFactory to MetadataFactory.cs
* 3278 Refactored DataSourceFactory to not be static. Added IDataSourceFactory interface.
* 3278 Refactored IKustoIntellisenseHelper.cs to not be static. Added IKustoIntellisenseHelper.cs interface.
* 3278 Removed unused functions from Scripter and deleted ScripterCore.cs because it was unused. Refactored Scripter.cs methods to not be static and created IScripter.cs
* 3278 Refactored datsasourceConnectionFactory in ConnectionService to use MEF through the InitializeService function
* 3278 Refactored IAutoCompleteHelper to not be static and added IAutoCompleteHelper interface.
* 3278 Removed unused classes DatabaseFullAccessException and FeatureWithFullDbAccess. Refactored ObjectExplorerService to use ImportingConstructor attribute. Removed commented out in KustoResultsReader. Removed unused functions from DatabaseLocksManager. Removed unused IDataSourceConnectionFactory from ConnectionService
* 3278 Moved SqlConnectionOpener class to new file. Added new interfaces IConnectedBindingQueue.cs and ISqlConnectionOpener.cs Added sqlConnectionOpener to dependency in ConnectedBindingQueue.
* 3278 Removed needsMetadata param from ConnectedBindingQueue. Added param to AddConnectionContext where it's used
* 3278 Refactored ConnectedBindingQueue to use MEF. Refactored usages to run against interface.
* 3278 Corrected ServiceHost namespace. Removed unused dependency to sqlToolsContext in LanguageService. Updated dependency in MetadataService and LanguageService to IProtocolEndpoint instead of ServiceHost
* 3278 Added back NextResult function and summary to KustoResultsReader. Renamed instance to _instance in ConnectionService and DatabaseLocksManager to stay consistent. Changed OpenDataSourceConnection to private in ConnectionService
* 3278 Converted helper methods back to static and removed backing interfaces
* 3278 Reverted AdminService > InitializeService to use ServiceHost as param
