* Added AzureMonitor to Microsoft.Kusto.ServiceLayer.
* Added Intellisense for AzureMonitor. Moved Intellisense logic from KustoIntellisenseClient to IntellisenseClientBase.
* Added ServiceName as a command parameter for starting Kusto.
* Added check to return null if connectionInfo is not in the connectionService.
* Added support for Dashboard in MetadataService and AdminService.
* Removed workspace id from databaseName for Monitor. Added logic for MetadataService and AdminService to return different information for AzureMonitor.
* Moved providerName and providerDescription to DataSourceFactory.
* Changed DatabaseName to include Name and Id. Changed ProviderName to LOGANALYTICS in DataSourceFactory
* Fixed unit tests
* Changed logic to use ServiceName instead of server to determine DataSourceType
* Code review feedback and reverted changes to ObjectExplorerService.
* Removed unused reference from HostLoader
* Changed Parallel.Invoke to await Task.Run
* Moved Kusto datasource and supporting classes to separate directory.
* Removed unused datasourceFactory from ConnectionService. Added GetDatabases and GetDatabaseInfo to IDataSource and child classes
* Renamed Instance variables in ObjectExplorerService. Removed unused attribute on TSqlFormatterService. Removed invalid comment in ConnectionService.
* Fixed warnings in build.
* Moved SizeInMB to DatabaseMetadata. Refactored ConvertToDatabaseInfo
* Fixed unit test
* added other languages to ServiceLayer
* fixed travis and appveyor yaml, added other languages for localization
* added spanish and german strings to credentialslocalization, also WIP for coreservices test
* added WIP localization test for CoreServices
* Removed unit test for coreservices, need to work on those later.
* restored travis and appveyor
* added kusto string resource localization test
* added localization test and locale change for SqlToolsCredentials
* added locale command options to ResourceProvider
* revert resourceprovider test
* Localized SqlTools.Credentials
* added hosting localization
* replaced new with translation to be consistent with main service layer
* reverted external changes as it is unnecessary
* Translated ManagedBatchParser
* test culture value
* test culture change
* temporarily comment out service layer command options
* returned sr.cs
* temporarily remove bad tests
* restored tests and allowed for SR culture to change for dependencies (for future testing)
* localization for resourceprovider components added.
* Set LocaleSetter to public and also removed InternalsVisible for Kusto, Credentials and ResourceProvider
* removed unnecessary changes
* Removed CredentialsCommandOptions
* renamed pt-BR to pt-br
* Rename sr.pt-BR.xlf.template to sr.pt-br.xlf.template
* Rename src/Microsoft.SqlTools.ServiceLayer/Localization/LCL/pt-BR/sr.resx.lcl to src/Microsoft.SqlTools.ServiceLayer/Localization/LCL/pt-br/sr.resx.lcl
* Rename sr.pt-BR.xlf to sr.pt-br.xlf
* Rename sr.pt-BR.resx to sr.pt-br.resx
* Rename sr.pt-BR.resx to sr.pt-br.resx
* Rename sr.pt-BR.xlf to sr.pt-br.xlf
* Rename sr.pt-BR.resx to sr.pt-br.resx
* Rename sr.pt-BR.xlf to sr.pt-br.xlf
* Rename sr.pt-BR.resx to sr.pt-br.resx
* Rename sr.pt-BR.xlf to sr.pt-br.xlf
* Rename sr.pt-BR.resx to sr.pt-br.resx
* Rename sr.pt-BR.xlf to sr.pt-br.xlf
* Rename sr.pt-BR.xlf to sr.pt-br.xlf
* Rename sr.pt-BR.resx to sr.pt-br.resx
* Rename sr.pt-BR.xlf to sr.pt-br.xlf
* Rename sr.pt-BR.resx to sr.pt-br.resx
* restored newlines
* Update sr.pt-br.xlf.template
Removed newline
* removed newline
* fixing newlines for consistency
* removed unnecessary newline
* Copy smoModel some rename
* Copy entire service layer
* Building copy
* Fixing some references
* Launch profile
* Resolve namespace issues
* Compiling tests. Correct manifest.
* Fixing localization resources
* ReliableKustoClient
* Some trimming of extra code and Kusto code
* Kusto client creation in bindingContent
* Removing Smo and new Kusto classes
* More trimming
* Kusto schema hookup
* Solidying DataSource abstraction
* Solidifying further
* Latest refatoring
* More refactoring
* Building and launching Kusto service layer
* Working model which enumerates databases
* Refactoring to pass IDataSource to all tree nodes
* Removing some dependencies on the context
* Working with tables and schema
* Comment checkin
* Refactoring to give out select script
* Query created and sent back to ADS
* Fix query generation
* Fix listing of databases
* Tunneling the query through.
* Successful query execution
* Return only results table
* Deleting Cms
* Delete DacFx
* Delete SchemaCompare and TaskServices
* Change build definition to not stop at launch
* Fix error after merge
* Save Kusto results in different formats (#935)
* save results as csv etc
* some fixes
Co-authored-by: Monica Gupta <mogupt@microsoft.com>
* 2407 Added OrderBy clause in KustoDataSource > GetDatabaseMetaData and GetColumnMetadata (#959)
* 2405 Defaulted Options when setting ServerInfo in ConnectionService > GetConnectionCompleteParams (#965)
* 2747 Fixed IsUnknownType error for Kusto (#989)
* 2747 Removed unused directives in Kusto > DbColumnWrapper. Refactored IsUnknownType to handle null DataTypeName
* 2747 Reverted IsUnknownType change in DbColumnWrapper. Changed DataTypeName to get calue from ColumnType. Refactored SafeGetValue to type check before hard casting to reduce case exceptions.
* Added EmbeddedResourceUseDependentUponConvention to Microsoft.Kusto.ServiceLayer.csproj. Also renamed DACfx to match Microsoft.SqlTools.ServiceLayer. Added to compile Exclude="**/obj/**/*.cs"
* Srahman cleanup sql code (#992)
* Removed Management and Security Service Code.
* Remove FileBrowser service
* Comment why we are using SqlServer library
* Remove SQL specific type definitions
* clean up formatter service (#996)
Co-authored-by: Monica Gupta <mogupt@microsoft.com>
* Code clean up and Kusto intellisense (#994)
* Code clean up and Kusto intellisense
* Addressed few comments
* Addressed few comments
* addressed comments
Co-authored-by: Monica Gupta <mogupt@microsoft.com>
* Return multiple tables for Kusto
* Changes required for Kusto manage dashboard (#1039)
* Changes required for manage dashboard
* Addressed comments
Co-authored-by: Monica Gupta <mogupt@microsoft.com>
* 2728 Kusto function support (#1038)
* loc update (#914)
* loc update
* loc updates
* 2728 moved ColumnInfo and KustoResultsReader to separate files. Added Folder and Function to TreeNode.cs
* 2728 Added FunctionInfo. Added Folder to ColumnInfo. Removed partial class from KustoResultsReader. Set Function.IsAlwaysLeaf=true in TreeNode.cs. In KustoDataSource changed tableMetadata type to TableMetaData. Added folder and function dictionaries. Refactored GetSchema function. Renamed GenerateColumnMetadataKey to GenerateMetadataKey
* 2728 Added FunctionInfo. Added Folder to ColumnInfo. Removed partial class from KustoResultsReader. Set Function.IsAlwaysLeaf=true in TreeNode.cs. In KustoDataSource changed tableMetadata type to TableMetaData. Added folder and function dictionaries. Refactored GetSchema function. Renamed GenerateColumnMetadataKey to GenerateMetadataKey
* 2728 Created new SqlConnection within using block. Refactored KustoDataSource > columnmetadata to sort on get instead of insert.
* 2728 Added GetFunctionInfo function to KustoDataSource.
* 2728 Reverted change to Microsoft.Kusto.ServiceLayer.csproj from merge
* 2728 Reverted change to SqlTools.ServiceLayer\Localization\transXliff
* 2728 Reverted change to sr.de.xlf and sr.zh-hans.xlf
* 2728 Refactored KustoDataSource Function folders to support subfolders
* 2728 Refactored KustoDataSource to use urn for folders, functions, and tables instead of name.
* Merge remote-tracking branch 'origin/main' into feature-ADE
# Conflicts:
# Packages.props
* 2728 Moved metadata files into Metadata subdirectory. Added GenerateAlterFunction to IDataSource and DataSourceBase.
* 2728 Added summary information to SafeAdd in SystemExtensions. Renamed local variable in SetTableMetadata
* 2728 Moved SafeAdd from SystemExtensions to KustoQueryUtils. Added check when getting database schema to return existing records before querying again. Added AddRange function to KustoQueryUtils. Created SetFolderMetadataForFunctions method.
* 2728 Added DatabaseKeyPrefix to only return tables to a database for the dashboard. Added logic to store all database tables within the tableMetadata dictionary for the dashboard.
* 2728 Created TableInfo and moved info objects into Models directory. Refactored KustoDataSource to lazy load columns for tables. Refactored logic to load tables using cslschema instead of schema.
* 2728 Renamed LoadColumnSchema to GetTableSchema to be consistent.
Co-authored-by: khoiph1 <khoiph@microsoft.com>
* Addressed comments
Co-authored-by: Shafiq Rahman <srahman@microsoft.com>
Co-authored-by: Monica Gupta <mogupt@microsoft.com>
Co-authored-by: Justin M <63619224+JustinMDotNet@users.noreply.github.com>
Co-authored-by: rkselfhost <rkselfhost@outlook.com>
Co-authored-by: khoiph1 <khoiph@microsoft.com>
* Update form .NET Core 2.2 to .NET Core 3.1
- Global variable for projects
- Change TFMs from netcoreapp2.2 to netcoreapp3.1
- Update global.json
- Update build.json
- Remove direct framework cake TestCore task
- Update travis dotnet version
- Update azure pipline file
- Update vscode launch.json
* Add Central Package Management
* Fix xUnit Breaking Change for MemberData type
* Fix xUnit breaking change for duplicate test method name
* Fix Rang/Index type conflict with System.Rang/Index
* Update vscode tasks.json
* Change serviceHostExecutable path in ServiceTestDriver.cs
* Downgrade SDK version (https://github.com/appveyor/ci/issues/3440)
- Appveyor hasn't installed latest SDK therefore I downgrade it until they install it.
* Dump Microsoft.SqlServer.DACFx
* Revert "Update SMO (#875)"
This reverts commit 96593d66e3.
* Revert "Switch Tools Service to new SMO and Microsoft.Data.SqlClient driver (#865)"
This reverts commit 9d140b53f3.
* Bump SMO to 160.1910315.0-preview
* switch to ambient props and targets files
* build against Microsoft.Data.SqlClient
* build tests
* fix test bug
* temporarily add SMO nuget to the repo
* update to released Microsoft.Data package
* Part1 : Changes to make cmdcmd script to work with parameters in script
* Stop SQL intellisense for SQLCMD
* Adding test for Intellisense handling of SQLCMD page
* Removing unintentional spacing changes caused by formatting
* Updating with smaller CR comments. Will discuss regarding script vs other options in batch info
* Removing unintentional change
* Adding latest PR comments
* Fix serialization tests & don't block thread
- Fixed potential null ref when closing streams
- Always clean up serialization queue if an error occurs
- Stop blocking dispatcher thread by not awaiting task that processes the message
- Improved error logging in EventFlowValidator to help debug issues
- Close stream on exception
* Adding ICompletionExtension interface
* Adding extension loading and execution logic
* Fixing compilation error in VS 2017
* Using MEF for completion extension discovery
* using await on GetCompletionItems
* Adding an integration test for completion extension and update the completion extension interface
* Update the completion extension test
* Fix issues based on review comments
* Remove try/cache based on review comments, fix a integration test.
* More changes based on review comments
* Fixing SendResult logic for completion extension loading
* Only load completion extension from the assembly passed in, add more comments in the test
* Adding right assert messages in the test.
* More fixes based on review comments
* Dropping ICompletionExtensionProvider, load assembly only if it's loaded at the first time or updated since last load.
* Fix based on the latest review comments
* Adding missing TSQL functions in default completion list
* Update jsonrpc documentation for completion/extLoad
This changes adds the following two notifications from the results processing within a batch. These new notifications allows a consumer to stream results from a resultset instead of getting them all at once after the entire resultset has been fetched.
ResultsAvailable
This is issued after at least 1 row has been fetched for this resultset.
ResultsUpdated
This is issued periodically as more rows are available on this resultset. The final send of this notification when all rows have been fetched has the property 'Complete' set to true in the ResultSummary object.
Detailed Change Log:
* Initial completed implementation of QueryResults stream feature. 3 unittests still need fixing
* Fix for the 3 failing test. I will look into making MockBehavior strict again for the three tests later
* Making GetReader/GetWriter use filestream objects in FileShare.ReadWrite mode so the file can be concurrently read and written
* Changing resultsAvailable also to fire off on a timer instead of after 1st row
* adding a project for clr TableValuedFunction to produce result set with delays after each row. This is helpful in end to end testing.
* Fixing up some tests and simplifying implementation of result update timer
* Address review comments
* Some test fixes
* Disabled flaky test verification
* Switch to the unified SMO NuGet so all binaries are strong named and signed.
* use one AssemblyLoader instance for all loads
* Revert "use one AssemblyLoader instance for all loads"
This reverts commit 48c59ffd5c57152de281c87acdbcad7ddf7ab760.
* Stop creating multiple AssemblyLoadContext objects during composition, per https://github.com/dotnet/coreclr/issues/19632
* restore high entropyva property
* Improve the comment
This change modifies the logging framework within sqltoolservice.
Moves away from custom Logger object to start using .Net tracing framework. It supports for the static Trace and TraceSource way of logging. For all new code it is recommend that we log the log messages using the existing static Logger class, while the code changes will continue to route the older Trace.Write* calls from the process to same log listeners (and thus the log targets) as used by the Logger class. Thus tracing in SMO code that uses Trace.Write* methods gets routed to the same file as the messages from rest of SQLTools Service code.
Make changes to start using .Net Frameworks codebase for all logging to unify our logging story.
Allows parameter to set tracingLevel filters that controls what kinds of message make it to the log file.
Allows a parameter to set a specific log file name so if these gets set by external code (the UI code using the tools service for example) then the external code is aware of the current log file in use.
Adding unittests to test out the existing and improved logging capabilities.
Sequences of checkins in development branch:
* Saving v1 of logging to prepare for code review. Minor cleanup and some end to end testing still remains
* Removing local launchSettings.json files
* added support for lazy listener to sqltoolsloglistener and removed incorrect changes to comments across files in previous checkin
* Converting time to local time when writing entries to the log
* move the hosting.v2 to new .net based logging code
* removing *.dgml files and addding them to .gitignore
* fixing typo of defaultTraceSource
* Addressing pull request feedback
* Adding a test to verify logging from SMO codebase
* propogating changes to v1 sqltools.hosting commandoptions.cs to the v2 version
* Fixing comments on start and stop callstack methods and whitespaces
* Commenting a test that got uncommented by mistake
* addding .gitattributes file as .sql file was observed to be misconstrued as a binary file
* Port v2 of Hosting service to SqlToolsService
- Renamed project to .v2 so that existing hosted service isn't impacted
- Copied over the CoreServices project which contains ConnectionServiceCore and other reusable services for anything interacting with MSSQL
- Ported unit test project across and verified tests run.
* Nuget package support for reusable DLLs
* Use 1.1 version per Karl's suggestion
* Use correct license URL and project URL
* Use new SMO packages
* Refactoring sql script formatting helpers into To and From helpers
* Updates to make error messages for formatting errors more useful
* Fixing dumb breaks in unit tests
* Addressing comments from PR
* Updates to the SR files...
* WIP
* All the new RowCreate tests are working
* Fixing a couple bugs with the row delete and row update tests
* Regenerating localization files
* Fixing multiple iteration in tests
* Fix to make sql exceptions surface properly to user (with important notes!)
* Adding support for detecting column size issues when updating a cell
* Adding unit tests for the exception on read scenario
* Multi-thread server lookup to increase perf for multiple subscriptions
* Use parallel execution for listsubscriptions
- Also refactored parallel execution code to be a bit more generic
Implementation of the resource provider APIs in order to support Create Firewall Rule. Provides definition for a ResourceProvider and Authentication service. The ResourceProvider supports firewall rules for now, and since authentication is routed through that method it will call into the auth service to set up the current account to be used.
Additional notes:
- Fixed deserialization by adding an Accept header. This shouldn't be necessary, but for some reason the firewall rule defaults to XML without this
- Use generic server list and parse the ID to get the resource group, avoiding a large number of extra calls for each RG
- Errors now include error message from the API
Porting of the vast majority of Azure-related code from SSDT. This is very large, so I want to put this out as one large "lift and shift" PR before I do the tools-service specific JSON-RPC service handlers, connect a new account handler (as the code to get necessary info from accounts and subscriptions isn't fully complete) and add tests over these
**What's in this PR**:
- Created 3 new projects:
- Microsoft.SqlTools.ResourceProvider will host the executable that accepts requests for Azure-related actions over the JSON-RPC protocol. This must be separate from other DLLs since a direct dependency on the Azure SDK DLLs fails (they're NetStandard 1.4 and you can't reference them if you have RuntimeIdentifiers in your .csproj file)
- Microsoft.SqlTools.ResourceProvider.Core is where all the main business logic is, including definitions and logic on how to navigate over resources and create firewall rules, etc.
- Microsoft.SqlTools.ResourceProvider.DefaultImpl is the actual Azure implementation of the resource provider APIs. The reason for separating this is to support eventual integration back into other tools (since their Azure and Identity services will be different).
- Implemented the AzureResourceManager that connects to Azure via ARM APIs and handles creating firewall rule and querying databases. The dependent DLLs have had major breaking changes, so will need additional verification to ensure this works as expected
- Ported the unit tests for all code that was not a viewmodel. Viewmodel test code will be ported in a future update as we plumb through a service-equivalent to these. Also, the DependencyManager code which has overlap with our service provider code is commented out. Will work to uncomment in a future update as it has value to test some scenarios
**What's not in this PR**:
- Identity Services. We currently just have a stub for the interface, and even that will likely change a little
- anything JSON-RPC or registered service related. These will be adapted from the viewmodels and added in a separate PR
* scripting working with race conditions
* new service works with no race conditions
* use new scripting service and commented out tests
* refactored peek definition to use mssql-scripter
* fixed peek definition tests
* removed auto gen comment
* fixed peek definition highlighting bug
* made scripting async and fixed event handlers
* fixed tests (without cancel and plan notifs)
* removed dead code
* added nuget package
* CR comments + select script service implementation
* minor fixes and added test
* CR comments and script select
* added unit tests
* code review comments and cleanup
* Bump SMO to 140.2.5 to pick-up private XEvent binaries
* Pick up SMO binaries from the build lab
* Add ProfilerService class placeholder
* Update SMO nuget package to include DB Scoped XEvents
* Stage changes
* Stage changes
* Update SMO to use RTM dependencies and remove separate SqlScript package
* Stage changes
* Iterate on profiler service
* Fix post-merge break in localization
* More refactoring
* Continue iterating on profiler
* Add test profiler listener
* Address a couple of the code review feedback
* Fix AppVeyor build break
* Use self-cleaning test file
* Initial changes to have serialization generate its own .exe
* Removed additional project from sln file
* remove all references to removed temporary project
* Moved shared contracts into own dll and fixed imports. Addressed PR comments
* Undid having a separate contracts project since that'll be a task for later on. Moved dbcellvalue and saveresultsrequest to Hosting, where they will be imported and shared by the service layer and serialization projects
* Switched backslashes in project reference in csproj file to forward slashes for consistency
* Moved necessary contracts back to service layer. Refactored CommandOptions to reduce code duplication. Addressed miscellaneous PR suggestions
* Accidentally left these files out of previous commit
* Initialized loggers for serialization and credentials with the logging directory provided by the cmd line arg, if there is one
* Changed default log directory paths for serialization and credentials. Removed unnecessary cast and added a copyright
* Changed name of generated executable for serialization service
* removed unnecessary object cast
* removing unnecessary imports and addressing other PR comments
* Initial .net core 2.0 conversion
* Convert a few more projects to .net core 2.0
* Convert a few more projects to .net core 2.0
* Fix build.cmd errors
* Add mising nuget package
* Remove dead code
* Add checked in references to workaround nuget package issues
* Update SLN file to refer to correct csproj files
* Rename applications to workaround .net core tooling bug
* Update nuget package with SQL Parser changes
* Add PreserveCompliationContext to avoid MEF bug
* Update smo version to pickup .net core 2 changes
* Pickup latest SMO changes to fix merge break
* Actually pickup correct SMO binaries
* Add support for SLES 12.2
* Fix break running archiving on Linux
* Revert "Add support for SLES 12.2"
This reverts commit 95cdb6d0e35a425be5c0081345d214079cbdc3db.
* Update to latest SMO build
* Install .Net Core 2 during install phase
* Move .Net Core install
* Try to reference dotnet.exe directly
* Fix code coverage script for CSPROJ instead of project.json
* Turn off test that is unreliable in AppVeyor builds.
* Fix appveyor.yml line feed.
* Turn off another flaky test failing in AppVeyor
