Commit Graph

28 Commits

Author SHA1 Message Date
Justin M
6f22dcf241 Kusto Auth Refactor Tests (#1148)
* Refactored Kusto.ServiceLayer to pass ConnectionDetails to DataSourceFactory instead of connection string. Created KustoConnectionDetails to map needed details to KustoClient.

* Removed unused ScriptingScriptOperation from KustoServiceLayer.

* Created DstsAuthenticationManager and moved logic for getting DstsToken. Updated error message for failing to create KustoConnection.

* Removed DstsAuthenticationManager.cs. Refactored DataSourceFactory to retrieve UserToken from ConnectionDetails.

* Renamed AzureAccountToken in ConnectionDetails to AccountToken. Changed mapping to KustoConnectionDetails based on the AccountToken.

* Removed Kusto.Data reference from ConnectionService and ScriptingListObjectsOperation. Moved creation of KustoConnectionStringBuilder to DataSourceFactory

* Added accountToken validation to DataSourceFactory Create.

* Renamed KustoConnectionDetails to DataSourceConnectionDetails. Renamed AzureToken to AuthToken.

* Refactored SchemaState and intellisense out of KustoClient to KustoIntellisenseClient. Added IIntellisenseClient. Added unit tests for KustoIntellisenseClient and KustoClient.

* Removed unused property dataSourceFactory from LanguageService > InitializeService. Moved KustoIntellisense functions from KustoIntellisenseHelper to KustoIntellisenseClient and made SchemaState private. Added IIntellisenseClient to IDataSource.

* Renamed directory from DataSourceIntellisense to Intellisense and updated namespace. Fixed namespace in ScriptDocumentInfo.
2021-01-29 11:25:23 -08:00
Justin M
f0a5e11d51 dSTS Authentication (#1125)
* Refactored Kusto.ServiceLayer to pass ConnectionDetails to DataSourceFactory instead of connection string. Created KustoConnectionDetails to map needed details to KustoClient.

* Removed unused ScriptingScriptOperation from KustoServiceLayer.

* Created DstsAuthenticationManager and moved logic for getting DstsToken. Updated error message for failing to create KustoConnection.

* Removed DstsAuthenticationManager.cs. Refactored DataSourceFactory to retrieve UserToken from ConnectionDetails.

* Renamed AzureAccountToken in ConnectionDetails to AccountToken. Changed mapping to KustoConnectionDetails based on the AccountToken.

* Removed Kusto.Data reference from ConnectionService and ScriptingListObjectsOperation. Moved creation of KustoConnectionStringBuilder to DataSourceFactory

* Added accountToken validation to DataSourceFactory Create.

* Renamed KustoConnectionDetails to DataSourceConnectionDetails. Renamed AzureToken to AuthToken.
2021-01-14 13:49:09 -08:00
Justin M
fd84367008 4094 Kusto AAD Authentication Refactor (#1115)
* 4094 Upgraded Kusto.Data and Kusto.Language to 9.0.3. Refactored KustoServiceLayer to use KustoConnectionStringBuilder instead of Sql.

* 4094 Refactored KustoClient>Initialize to take connectionString as optional. Removed unused properties from ConnectionService>ParseConnectionString

* 4094 Updated Kusto.Data and Kusto.Language nuget packages to 9.0.4. Removed IsDedicatedAdminConnection function in ConnectionService.

* 4094 Fixed unit tests
2020-11-20 11:54:40 -08:00
Justin M
67c7e20c13 13159 Created SetDatabaseName in KustoClient to take database name from ( ). Changed ListDatabases in Kusto ConnnectionService to set DatabaseNames in a different format when the PrettyName doesn't match the name. (#1123) 2020-11-19 14:18:58 -08:00
Justin M
30d600e423 Kusto Client Cleanup (#1102)
* Cleaned up KustoClient with formatting and switched logic between ExecuteCommandAsync and ExecuteQueryAsync to match their respective names.

* Refactored ExecuteControlCommandAsync to call ExecuteControlCommandAsync in KustoAdminProvider. Fixed spacing in GetClientRequestProperties

* Refactored ExecuteQueryAsync to not use Task.Run
2020-11-17 10:26:09 -08:00
Justin M
954df47ec9 13159 Kusto Aria Fix (#1112)
* 13159 Changed NodeValue in TreeNode to PrettyName instead of Name. In KustoDataSource, changed Exists to run different query for Aria. Changed GetClusterDiagnostics to return empty DiagnosticsInfo for Aria. Changed SetDatabaseMetadata to not includeSizeDetails for Aria connections.

* 13159 Fixed unit test Refresh_Returns_Children
2020-11-02 15:35:37 -08:00
Monica Gupta
dbfcf4ad61 Fix for queries with comments separated by new lines (#1101)
* Fix for queries with comments separated by new lines

* Mimic Kusto behaviour for comments ran as query

* extra check

* address comment

Co-authored-by: Monica Gupta <mogupt@microsoft.com>
2020-10-20 13:53:34 -07:00
Monica Gupta
1ee0c1d09a Make Kusto control commands work (#1094)
Co-authored-by: Monica Gupta <mogupt@microsoft.com>
2020-10-08 17:51:08 -07:00
Justin M
a94ab9d8b4 Changed logic to use database name provided by connectionString (#1091) 2020-10-06 20:45:54 -07:00
Justin M
f2eb590d97 3490 Kusto Connection Refresh Fix (#1085)
* 3490 Injected OwnerUri into KustClient to store for token refreshing. Removed UpdateAzureToken from IDataSource, DataSourceBase, and KustoDataSource. Removed logic for retrying queries related to Unauthorized datasource in Batch and Query. Changed ScriptingService, ScriptingScriptOperation, and ScriptAsScriptingOperation to take DataSource in the constructor instead of datasourcefactory. Changed ScriptingService to inject ConnectionService through InitializeService function.

* 3490 Removed Catch block for DataSourceUnauthorizedException in ExecuteControlCommandAsync

* 3490 Removed OwnerUri from KustoClient and used azureAccountToken to refresh token in ConnectionService.

* 3490 Reverted unneeded changes.

* 3490 Split ExecuteQuery in KustoClient to execute first query then remaining queries after

* 3490 Passed OwnerUri down into KustoClient to refresh token.

* 3490 Removed DataSourceUnauthorizedException. Refactored ExecuteQuery to catch aggregate exception. Added RefreshAzureToken logic to ExecuteControlCommand

* 3490 Added logic to update ReliableDataSourceConnection azure token within ConnectionInfo.

* 3490 Add retry logic to ExecuteQuery and ExecuteControlCommand in KustoClient
2020-10-06 17:20:13 -07:00
Monica Gupta
164ca951da Fix for multiple queries separated by space for Kusto (#1089)
* Fix for multiple queries separated by space

* Addressed comments

Co-authored-by: Monica Gupta <mogupt@microsoft.com>
2020-10-04 19:12:33 -07:00
Justin M
d061e781f4 3644 Kusto Azure Token Refresh (#1081)
* 3644 Created KustoClient to handle interaction with Kusto servers. Refactored KustoDataSource and moved execution logic into KustoClient.

* 3644 Added RequestSecurityTokenParams, RequestSecurityTokenResponse, and SecurityTokenRequest to Kusto. Moved intellisense functions from KustoDataSource to KustoIntellisenseHelper. Added SchemaState to readonly property on KustoDataSource. Added catch block to Batch.cs to catch KustoRequestExceptions.

* 3644 Removed unused reference from ConnectionDetails and ConnectedBindingContext. Added UpdateAzureToken function to IKustoClient, KustoClient, IDataSource, KustoDataSource, and KustoDataSource. Added dataSource.Dispose to ReliableDataSourceConnection > Close. Added RefreshAzureToken to ConnectionService to refresh azure token.

* 3644 Removed unused properties from RequestSecurityTokenParams and RequestSecurityTokenResponse

* 3644 Added default to DbColumnWrapper to UnknownTypeName when null. Moved database query logic from KustoIntellisenseHelper to KustoClient. Moved KustoIntellisenseHelper data objects out of class. Changed SchemaState to load through separate tasks.

* 3644 Changed ReRunQuery logic in Kusto Batch to flip back to false if the query fails a second time so it can be rejected

* 3644 Updated GetAutoCompleteSuggestions and GetHoverHelp with changes from main

* 3644 Added AccountId to RequestSecurityTokenParams and set value in ConnectionService. Added throw to Batch.cs to ensure exceptions that are non-Unauthorized bubble up to ADS.

* 3644 Changed KustoUnauthorizedException to take original exception as inner exception. Changed catch block to only throw KustoUnauthorizedException when FailureCode is 401

* 3644 Renamed KustoUnauthorizedException to DataSourceUnauthorizedException. Moved logic to throw exception down into KustoClient. Changed retryLogic in Batch.cs to a decrementing count

* 3644 Changed logic in Batch.cs for throwing InvalidOperationException
2020-09-28 13:24:23 -07:00
Monica Gupta
fd4d067e71 Fix intellisense if multiple kusto queries exist (#1080)
Co-authored-by: Monica Gupta <mogupt@microsoft.com>
2020-09-21 10:23:00 -07:00
Shafiq Ur Rahman
2e6b134953 Result handling for Kusto results (#1079)
* Revert "Fix for displaying multiple table (#1075)"

This reverts commit df0be31019.

* KustoResultsReader trims results
2020-09-21 10:02:23 -07:00
Monica Gupta
fea050448d Fix to escape table names only (#1078)
Co-authored-by: Monica Gupta <mogupt@microsoft.com>
2020-09-20 17:19:01 -07:00
rajeshka
df0be31019 Fix for displaying multiple table (#1075) 2020-09-17 11:59:01 -07:00
Justin M
c80b70bea4 3715 Default database name for Kusto (#1076)
* 3715 Refactored GetDatabaseName to load database names from kusto server when no database name available in connection string.

* 3715 Removed variable name from GetDatabaseName
2020-09-15 13:34:21 -07:00
Monica Gupta
f25fb3c5d7 Kusto control command with comments or spaces (#1074)
Co-authored-by: Monica Gupta <mogupt@microsoft.com>
2020-09-12 00:10:48 -07:00
Justin M
d39a67df30 3643 Intellisense fix for spaces (#1073)
* 3643 Enhanced GetAutoCompleteSuggestions to escape spaces in the insertText when generating list of suggestions. Changed detail to include kind.

* 3643 Fixed formatting

* 3643 Changed variable in GetAutoCompleteSuggestions
2020-09-11 15:22:58 -07:00
Monica Gupta
66cba8b77d Fix suggestions after keyword (#1071)
Co-authored-by: Monica Gupta <mogupt@microsoft.com>
2020-09-10 09:52:26 -07:00
Justin M
f0ec7e14d4 3702 Changed CancelQuery to use ExecuteControlCommand instead of ExecuteQuery in KustoDataSource. Fixed runtime error in NodePathGenerator for loading TreeNodeDefinition.xml (#1069) 2020-09-08 14:05:45 -07:00
Justin M
cd1e4f5ec5 3647 Kusto Refresh Logic (#1067)
* 3647 Renamed ParentMetadata to DatabaseMetadata in FolderMetadata.cs. Removed Refresh function from IDataSource and DataSourceBase since it's not used outside of KustoDataSource. Added unit tests for ServerNode.cs. Removed unused constructor from TreeNode and unused AddChild function.

* 3647 Refactored KustoDataSource. Moved Set logic for DatabaseMetadata and columns into separate functions

* 3647 Reverted Rename of ParentMetadata in FolderMetadata. Reverted removal of Refresh function from DataSourceBase and IDatasource

* 3647 Renamed SmoModel directories to DataSourceModel in Kusto projects.

* 3647 Removed reference to Kusto.ServiceLayer in SqlTools.ServiceLayer
2020-09-08 08:01:39 -07:00
Justin M
1577177153 3278 Kusto Unit Tests - Part 1 (#1057)
* 3278 Added unit tests in MetadataFactoryTests and Microsoft.Kusto.ServiceLayer.UnitTests project

* 3278 Removed todo and changed unit test to validate megabytes

* 3278 Added file and unit tests in AutoCompleteHelperTests.cs

* 3278 Removed unused functions from Kusto > ScriptAsScriptingOperation

* 3278 Added unit tests for DataSourceFactory

* 3278 Refactored AdminService to pass in ConnectionService rather than through instance variable. Added unit test for AdminServiceTests

* 3278 Refactored DataSourceFactory to not have static functions for future unit tests

* 3278 Re-added properties that were flagged as unused but are being used by ADS in ReliableDataSourceConnection.cs

* 3278 Re-added properties that were flagged as unused but are being used by ADS in ReliableDataSourceConnection.cs

* adding pipeline to execute tests (#1062)

* 3278 Converted GetDefaultAutoComplete and GetDefaultSemanticMarkers to static functions in DataSourceFactory. Removed unused constructor in ScriptFile. Added positive unit tests for both functions

* undoing release version bump

* adding additional configs

* adressing feedback

* Correcting path in csproj

Co-authored-by: Jorge Berumen <52225468+joberume@users.noreply.github.com>
Co-authored-by: joberume <jberumen3@miners.utep.edu>
2020-08-31 11:11:12 -07:00
rajeshka
0b905fef75 Reversing the multi-table fix till we have proper fix (#1060) 2020-08-27 20:26:31 -07:00
Justin M
26cc913c61 3491 Kusto Execute Function support (#1055)
* 3491 Added GenerateExecuteFunctionScript to IDataSource, DataSourceBase, and KustoDataSource. Added ExecutionFunction to Scripter and IScripter.

* 3491 Refactored GenerateScriptForFunction in ScriptAsScriptingOperation to handle execute and alter
2020-08-26 14:45:21 -07:00
Justin M
3f725b5aec 3487 Created new function GetTablesForDashboard in KustoDataSource to load tables for ADS Dashboard (#1056) 2020-08-25 11:55:07 -07:00
Justin M
14f5a3e0f1 3278 Kusto Unit Test Refactor (#1053)
* 3278 Moved functions related to Metadata from DataSourceFactory to MetadataFactory.cs

* 3278 Refactored DataSourceFactory to not be static. Added IDataSourceFactory interface.

* 3278 Refactored IKustoIntellisenseHelper.cs to not be static. Added IKustoIntellisenseHelper.cs interface.

* 3278 Removed unused functions from Scripter and deleted ScripterCore.cs because it was unused. Refactored Scripter.cs methods to not be static and created IScripter.cs

* 3278 Refactored datsasourceConnectionFactory in ConnectionService to use MEF through the InitializeService function

* 3278 Refactored IAutoCompleteHelper to not be static and added IAutoCompleteHelper interface.

* 3278 Removed unused classes DatabaseFullAccessException and FeatureWithFullDbAccess. Refactored ObjectExplorerService to use ImportingConstructor attribute. Removed commented out in KustoResultsReader. Removed unused functions from DatabaseLocksManager. Removed unused IDataSourceConnectionFactory from ConnectionService

* 3278 Moved SqlConnectionOpener class to new file. Added new interfaces IConnectedBindingQueue.cs and ISqlConnectionOpener.cs Added sqlConnectionOpener to dependency in ConnectedBindingQueue.

* 3278 Removed needsMetadata param from ConnectedBindingQueue. Added param to AddConnectionContext where it's used

* 3278 Refactored ConnectedBindingQueue to use MEF. Refactored usages to run against interface.

* 3278 Corrected ServiceHost namespace. Removed unused dependency to sqlToolsContext in LanguageService. Updated dependency in MetadataService and LanguageService to IProtocolEndpoint instead of ServiceHost

* 3278 Added back NextResult function and summary to KustoResultsReader. Renamed instance to _instance in ConnectionService and DatabaseLocksManager to stay consistent. Changed OpenDataSourceConnection to private in ConnectionService

* 3278 Converted helper methods back to static and removed backing interfaces

* 3278 Reverted AdminService > InitializeService to use ServiceHost as param
2020-08-24 13:18:00 -07:00
Monica Gupta
148b6e398d Added new Kusto ServiceLayer (#1009)
* Copy smoModel some rename

* Copy entire service layer

* Building copy

* Fixing some references

* Launch profile

* Resolve namespace issues

* Compiling tests. Correct manifest.

* Fixing localization resources

* ReliableKustoClient

* Some trimming of extra code and Kusto code

* Kusto client creation in bindingContent

* Removing Smo and new Kusto classes

* More trimming

* Kusto schema hookup

* Solidying DataSource abstraction

* Solidifying further

* Latest refatoring

* More refactoring

* Building and launching Kusto service layer

* Working model which enumerates databases

* Refactoring to pass IDataSource to all tree nodes

* Removing some dependencies on the context

* Working with tables and schema

* Comment checkin

* Refactoring to give out select script

* Query created and sent back to ADS

* Fix query generation

* Fix listing of databases

* Tunneling the query through.

* Successful query execution

* Return only results table

* Deleting Cms

* Delete DacFx

* Delete SchemaCompare and TaskServices

* Change build definition to not stop at launch

* Fix error after merge

* Save Kusto results in different formats (#935)

* save results as csv etc

* some fixes

Co-authored-by: Monica Gupta <mogupt@microsoft.com>

* 2407 Added OrderBy clause in KustoDataSource > GetDatabaseMetaData and GetColumnMetadata (#959)

* 2405 Defaulted Options when setting ServerInfo in ConnectionService > GetConnectionCompleteParams (#965)

* 2747 Fixed IsUnknownType error for Kusto (#989)

* 2747 Removed unused directives in Kusto > DbColumnWrapper. Refactored IsUnknownType to handle null DataTypeName

* 2747 Reverted IsUnknownType change in DbColumnWrapper. Changed DataTypeName to get calue from ColumnType. Refactored SafeGetValue to type check before hard casting to reduce case exceptions.

* Added EmbeddedResourceUseDependentUponConvention to Microsoft.Kusto.ServiceLayer.csproj. Also renamed DACfx to match Microsoft.SqlTools.ServiceLayer. Added to compile Exclude="**/obj/**/*.cs"

* Srahman cleanup sql code (#992)

* Removed Management and Security Service Code.

* Remove FileBrowser service

* Comment why we are using SqlServer library

* Remove SQL specific type definitions

* clean up formatter service (#996)

Co-authored-by: Monica Gupta <mogupt@microsoft.com>

* Code clean up and Kusto intellisense (#994)

* Code clean up and Kusto intellisense

* Addressed few comments

* Addressed few comments

* addressed comments

Co-authored-by: Monica Gupta <mogupt@microsoft.com>

* Return multiple tables for Kusto

* Changes required for Kusto manage dashboard (#1039)

* Changes required for manage dashboard

* Addressed comments

Co-authored-by: Monica Gupta <mogupt@microsoft.com>

* 2728 Kusto function support (#1038)

* loc update (#914)

* loc update

* loc updates

* 2728 moved ColumnInfo and KustoResultsReader to separate files. Added Folder and Function to TreeNode.cs

* 2728 Added FunctionInfo. Added Folder to ColumnInfo. Removed partial class from KustoResultsReader. Set Function.IsAlwaysLeaf=true in TreeNode.cs. In KustoDataSource changed tableMetadata type to TableMetaData. Added folder and function dictionaries. Refactored GetSchema function. Renamed GenerateColumnMetadataKey to GenerateMetadataKey

* 2728 Added FunctionInfo. Added Folder to ColumnInfo. Removed partial class from KustoResultsReader. Set Function.IsAlwaysLeaf=true in TreeNode.cs. In KustoDataSource changed tableMetadata type to TableMetaData. Added folder and function dictionaries. Refactored GetSchema function. Renamed GenerateColumnMetadataKey to GenerateMetadataKey

* 2728 Created new SqlConnection within using block. Refactored KustoDataSource > columnmetadata to sort on get instead of insert.

* 2728 Added GetFunctionInfo function to KustoDataSource.

* 2728 Reverted change to Microsoft.Kusto.ServiceLayer.csproj from merge

* 2728 Reverted change to SqlTools.ServiceLayer\Localization\transXliff

* 2728 Reverted change to sr.de.xlf and sr.zh-hans.xlf

* 2728 Refactored KustoDataSource Function folders to support subfolders

* 2728 Refactored KustoDataSource to use urn for folders, functions, and tables instead of name.

* Merge remote-tracking branch 'origin/main' into feature-ADE

# Conflicts:
#	Packages.props

* 2728 Moved metadata files into Metadata subdirectory. Added GenerateAlterFunction to IDataSource and DataSourceBase.

* 2728 Added summary information to SafeAdd in SystemExtensions. Renamed local variable in SetTableMetadata

* 2728 Moved SafeAdd from SystemExtensions to KustoQueryUtils. Added check when getting database schema to return existing records before querying again. Added AddRange function to KustoQueryUtils. Created SetFolderMetadataForFunctions method.

* 2728 Added DatabaseKeyPrefix to only return tables to a database for the dashboard. Added logic to store all database tables within the tableMetadata dictionary for the dashboard.

* 2728 Created TableInfo and moved info objects into Models directory. Refactored KustoDataSource to lazy load columns for tables. Refactored logic to load tables using cslschema instead of schema.

* 2728 Renamed LoadColumnSchema to GetTableSchema to be consistent.

Co-authored-by: khoiph1 <khoiph@microsoft.com>

* Addressed comments

Co-authored-by: Shafiq Rahman <srahman@microsoft.com>
Co-authored-by: Monica Gupta <mogupt@microsoft.com>
Co-authored-by: Justin M <63619224+JustinMDotNet@users.noreply.github.com>
Co-authored-by: rkselfhost <rkselfhost@outlook.com>
Co-authored-by: khoiph1 <khoiph@microsoft.com>
2020-08-12 15:34:38 -07:00