Commit Graph

787 Commits

Author SHA1 Message Date
Charles Gagnon
32d7a63d7c Send error result if parse error occurs during convert (#1121) 2020-11-16 13:48:30 -08:00
Aleksei Guzev
11960fd904 Fix handling some SQL Assessment warnings (#1114)
* Some new SQL Assessment warnings may have Check property set to null. Handle the warnings with no exception.
* Make names of SQL Assessment test methods more informative.
2020-11-04 22:13:46 +03:00
Benjin Dubishar
65c4fc01aa Adding External Streaming Job I/O validation to DacFxService (#1106)
* checkpoint

* Not having cake, nor eating it

* Working

* Swapping external dll for nupkg

* Extracting statement out of full TSQL

* Improving error message

* Fixing filename capitalization

* Reverting csproj changes

* Adding updated sr.cs file

* VS lost tracking on strings file?

* PR feedback

* resx additions

* More updated string files

* Swapped nuget for dll

* Revert "Swapped nuget for dll"

This reverts commit 6013f3fadf58ebc7e3590a46811d9fd9fc3eaa4a.

* Bumped netcore version to pull in support for extern aliasing nugets
2020-11-02 12:03:14 -08:00
Benjin Dubishar
271a0f5733 Updated loc build outputs (#1110) 2020-10-30 14:04:56 -07:00
BranislavGrbicMDCS
4ce9416219 removing server roles for sql od (#1107) 2020-10-30 08:15:29 -07:00
Cory Rivera
8c805a72a9 Add a parent process ID argument to ServiceLayer so that we can exit gracefully if the parent process exits. (#1103) 2020-10-29 13:00:23 -07:00
BranislavGrbicMDCS
3e98a2f17b [brgrbic] SqlOD changes for cloud Server Type and minor bug fixes (#1092)
* The ServerType of SQL OD in SMO is being changed to cloud. This is a minimal set of changes needed in sqltoolservice in order to keep everything working within ADS.

* SMO version update
2020-10-23 18:48:33 +02:00
Charles Gagnon
9b9325b43e Split sql batches into separate code cells for convert (#1100)
* Split sql batches into separate code cells for convert

* remove unneeded sort
2020-10-16 16:31:40 -07:00
Karl Burtram
b9e1618260 Revert "Rollback migration changes" (#1099)
* Revert "Revert "Initial migration service changes for assessment support (#1093)" (#1098)"

This reverts commit e96c0064ad.

* Move SQL Tools Service last in Main project list
2020-10-16 16:29:22 -07:00
Charles Gagnon
e96c0064ad Revert "Initial migration service changes for assessment support (#1093)" (#1098)
This reverts commit 1a0388c8cb.
2020-10-16 13:35:59 -07:00
Charles Gagnon
b64d5a3146 Fix Notebook Convert namespace (#1097) 2020-10-16 12:27:54 -07:00
Karl Burtram
1a0388c8cb Initial migration service changes for assessment support (#1093)
* Migration service WIP 1

* Assessment DLLs

* WIP

* WIP 2

* Add migration nuget package

* WIP2

* Fix up assessment handler

* WIP

* Fix build scripts

* WIP

* Update Assessment references

* Fix metadata copy to work with publish
2020-10-13 11:02:29 -07:00
Charles Gagnon
725d4ba5db Fix Notebook->Sql convert to correctly handle multiline comments within a batch (#1087) 2020-10-02 08:27:57 -07:00
Alex Ma
47e7a2f492 Fix for multiple parameter error. (#1082)
* Clearing first use of parameters

* restore space

* added finally block

* Added try catch block in correct position

* fixed spacing

* added space before try

* duplicate row check simplified by Alan

* Revert "duplicate row check simplified by Alan"

This reverts commit 445ac506bd96b353266778abd0cf9bad2be2a3c3.

* removed exception and changed message

* added exception back for test purposes

* added working executescalar

* Added "<=" for delete action that deletes nothing.

* spacing fix for check duplicate

* Added comments and changed function logic.

* added clarifying message

* Added new extended class

* small space fix
2020-09-25 10:52:59 -07:00
Justin M
cd1e4f5ec5 3647 Kusto Refresh Logic (#1067)
* 3647 Renamed ParentMetadata to DatabaseMetadata in FolderMetadata.cs. Removed Refresh function from IDataSource and DataSourceBase since it's not used outside of KustoDataSource. Added unit tests for ServerNode.cs. Removed unused constructor from TreeNode and unused AddChild function.

* 3647 Refactored KustoDataSource. Moved Set logic for DatabaseMetadata and columns into separate functions

* 3647 Reverted Rename of ParentMetadata in FolderMetadata. Reverted removal of Refresh function from DataSourceBase and IDatasource

* 3647 Renamed SmoModel directories to DataSourceModel in Kusto projects.

* 3647 Removed reference to Kusto.ServiceLayer in SqlTools.ServiceLayer
2020-09-08 08:01:39 -07:00
Karl Burtram
479e5242ff Wrong parameter usage in serialization (#1065) 2020-09-04 10:09:37 -07:00
Karl Burtram
3393e31f55 Remove kustoservicelayer from SQL Tools Service load (#1068) 2020-09-04 10:09:15 -07:00
Karl Burtram
5cf5b59a0d Insights generator project (#1066)
* InsightsGenerator project template files

* Add insights projects to SLN

* Setting up siggen class (#1003)

* Setting up siggen class

* fixed the learn method

* Refactoring code
Fixed compile errors

* renamed results to result

* Basic transformation logic (#1004)

* Fix a couple bugs and add a simple test (#1007)

* Fix a couple bugs and add a simple test

* More tests and bug fix

* Nara/workflow (#1006)

* added a queue processor

* ordered  using statements

* Armemon/analytics (#1008)

* Basic transformation logic

* changed some structure of siggen

* added sum and average method, as well as select rows by input name

* add insights to results

* min, max added

Co-authored-by: Karl Burtram <karlb@microsoft.com>
Co-authored-by: Aasim Khan <aasimkhan30@gmail.com>
Co-authored-by: Arslan Memon <armemon@microsoft.com>

* Added rules engine base implementation (#1005)

* Added rules engine base implementation

* update comments

* addressing comments

* adding template text to columnheaders object

* adding template text to columnheaders object

* fixing columnheaders class

* Added test

* Added Template Parser unit test in Test project

* Deleted unnecessary files and reverted the files that were modified by mistake

Co-authored-by: Jinjing Arima <jiarima@microsoft.com>

* Insights generator message handler placeholder (#1013)

* Aasim/insights/insight methods (#1014)

* Basic transformation logic

* changed some structure of siggen

* Added top and bottom insight functions

* Added top, bottom insights
Added tests for top, bottom insights

* Armemon/insights2 (#1011)

* max and min insightsperslice, and tests

* got rid of unneccesssary function

* get indexes

Co-authored-by: Arslan Memon <armemon@microsoft.com>

* Armemon/insights2 (#1012)

* max and min insightsperslice, and tests

* got rid of unneccesssary function

* get indexes

* learn for stringinputtyype

* add learn implentation

Co-authored-by: Arslan Memon <armemon@microsoft.com>

* Added Tests
Removed duplicate methods

Co-authored-by: Karl Burtram <karlb@microsoft.com>
Co-authored-by: arslan9955 <53170027+arslan9955@users.noreply.github.com>
Co-authored-by: Arslan Memon <armemon@microsoft.com>

* Armemon/insights2 (#1016)

* Basic transformation logic

* changed some structure of siggen

* Added top and bottom insight functions

* Added top, bottom insights
Added tests for top, bottom insights

* max and min insightsperslice, and tests

* got rid of unneccesssary function

* get indexes

* learn for stringinputtyype

* add learn implentation

* add unique inputs

* fix merge error

* add to result

Co-authored-by: Karl Burtram <karlb@microsoft.com>
Co-authored-by: Aasim Khan <aasimkhan30@gmail.com>
Co-authored-by: Arslan Memon <armemon@microsoft.com>

* Added all the templates (#1015)

* Added all the templates
Added a method to find matched template

* Added a function to replace # and ## values in a template

* Added ReplaceHashesInTemplate call

* Added comments

* Updated the template txt

* Updated GetTopHeadersWithHash function to add #toplist

* Updated the logic per offline discussion with Hermineh

* Update request handler contract to take array (#1020)

* added rulesengine findmatchingtemplate (#1019)

* Add support for getting DacFx deploy options from a publish profile (#995)

* add support for getting options from a publish profile

* update comments

* set values for default options if they aren't specified in the publish profile

* addressing comments

* Updating to latest DacFx for a bug fix (#1010)

* added rulesengine findmatchingtemplate

* Update DacFx deploy and generate script with options (#998)

* update deploy and generate script to accept deployment options

* add tests

* add test with option set to true

* merge

* merge

* incorporated FindMatchedTemplate

Co-authored-by: Kim Santiago <31145923+kisantia@users.noreply.github.com>
Co-authored-by: Udeesha Gautam <46980425+udeeshagautam@users.noreply.github.com>

* -Added logic for Insights Generator Service Handler (#1017)

* -Added logic for Insights Generator Service Handler

* Fixed some logic

* Adding workflow test

* Update transform and add tests (#1024)

* Jiarima/fix rules engine logic (#1021)

* Added all the templates
Added a method to find matched template

* Added a function to replace # and ## values in a template

* Added ReplaceHashesInTemplate call

* Added comments

* Updated the template txt

* Updated GetTopHeadersWithHash function to add #toplist

* Updated the logic per offline discussion with Hermineh

* Update with the fixes

* Updated template and foreach conditions

* Added distinct

* Updated tests according to the logic change (#1026)

* Nara/remove queing (#1023)

* loc update (#914)

* loc update

* loc updates

* Add support for getting DacFx deploy options from a publish profile (#995)

* add support for getting options from a publish profile

* update comments

* set values for default options if they aren't specified in the publish profile

* addressing comments

* Updating to latest DacFx for a bug fix (#1010)

* Update DacFx deploy and generate script with options (#998)

* update deploy and generate script to accept deployment options

* add tests

* add test with option set to true

* intermediate check in for merge, transformed not working

* intermediate check in for merge, transformed not working

* added test case

* merged

Co-authored-by: khoiph1 <khoiph@microsoft.com>
Co-authored-by: Kim Santiago <31145923+kisantia@users.noreply.github.com>
Co-authored-by: Udeesha Gautam <46980425+udeeshagautam@users.noreply.github.com>

* Output data types from transform (#1029)

* Fix bug process input_g (#1030)

* Fixed the insight generator service (#1028)

* Jiarima/added more testings (#1031)

* Added another test
Updated ReplaceHashesInTemplate function to return string instead of Template

* Added third test

* Merged

* Reverted the workflow file to match with the one in hack/insights

* Bugs fixes to hook insights up to ADS (#1033)

* Bug fixes for hack insights (#1032)

* Fixed the minColumn index bug in Data Transformation
Fixed the template matching logic.

* Adding changes from PR

* Try to fix Workflow tests

* Readd workflow tests

* Fix template load location

Co-authored-by: Aasim Khan <aasimkhan30@gmail.com>
Co-authored-by: Nara <NaraVen@users.noreply.github.com>
Co-authored-by: arslan9955 <53170027+arslan9955@users.noreply.github.com>
Co-authored-by: Arslan Memon <armemon@microsoft.com>
Co-authored-by: gadudhbh <68879970+gadudhbh@users.noreply.github.com>
Co-authored-by: Jinjing Arima <jiarima@microsoft.com>
Co-authored-by: jiarima <68882862+jiarima@users.noreply.github.com>
Co-authored-by: Kim Santiago <31145923+kisantia@users.noreply.github.com>
Co-authored-by: Udeesha Gautam <46980425+udeeshagautam@users.noreply.github.com>
Co-authored-by: khoiph1 <khoiph@microsoft.com>
2020-09-03 14:15:51 -07:00
Justin M
1577177153 3278 Kusto Unit Tests - Part 1 (#1057)
* 3278 Added unit tests in MetadataFactoryTests and Microsoft.Kusto.ServiceLayer.UnitTests project

* 3278 Removed todo and changed unit test to validate megabytes

* 3278 Added file and unit tests in AutoCompleteHelperTests.cs

* 3278 Removed unused functions from Kusto > ScriptAsScriptingOperation

* 3278 Added unit tests for DataSourceFactory

* 3278 Refactored AdminService to pass in ConnectionService rather than through instance variable. Added unit test for AdminServiceTests

* 3278 Refactored DataSourceFactory to not have static functions for future unit tests

* 3278 Re-added properties that were flagged as unused but are being used by ADS in ReliableDataSourceConnection.cs

* 3278 Re-added properties that were flagged as unused but are being used by ADS in ReliableDataSourceConnection.cs

* adding pipeline to execute tests (#1062)

* 3278 Converted GetDefaultAutoComplete and GetDefaultSemanticMarkers to static functions in DataSourceFactory. Removed unused constructor in ScriptFile. Added positive unit tests for both functions

* undoing release version bump

* adding additional configs

* adressing feedback

* Correcting path in csproj

Co-authored-by: Jorge Berumen <52225468+joberume@users.noreply.github.com>
Co-authored-by: joberume <jberumen3@miners.utep.edu>
2020-08-31 11:11:12 -07:00
Aleksei Guzev
07700560a6 Fix script generated for SQL Assessment results (#1058)
* INSERT VALUES has the limit of 1000 rows. Replace with derived table.
* Remove NOT NULL restriction from the generated table.
* Fix line endings in SQL Assessment source files.
2020-08-28 20:17:54 +03:00
Justin M
14f5a3e0f1 3278 Kusto Unit Test Refactor (#1053)
* 3278 Moved functions related to Metadata from DataSourceFactory to MetadataFactory.cs

* 3278 Refactored DataSourceFactory to not be static. Added IDataSourceFactory interface.

* 3278 Refactored IKustoIntellisenseHelper.cs to not be static. Added IKustoIntellisenseHelper.cs interface.

* 3278 Removed unused functions from Scripter and deleted ScripterCore.cs because it was unused. Refactored Scripter.cs methods to not be static and created IScripter.cs

* 3278 Refactored datsasourceConnectionFactory in ConnectionService to use MEF through the InitializeService function

* 3278 Refactored IAutoCompleteHelper to not be static and added IAutoCompleteHelper interface.

* 3278 Removed unused classes DatabaseFullAccessException and FeatureWithFullDbAccess. Refactored ObjectExplorerService to use ImportingConstructor attribute. Removed commented out in KustoResultsReader. Removed unused functions from DatabaseLocksManager. Removed unused IDataSourceConnectionFactory from ConnectionService

* 3278 Moved SqlConnectionOpener class to new file. Added new interfaces IConnectedBindingQueue.cs and ISqlConnectionOpener.cs Added sqlConnectionOpener to dependency in ConnectedBindingQueue.

* 3278 Removed needsMetadata param from ConnectedBindingQueue. Added param to AddConnectionContext where it's used

* 3278 Refactored ConnectedBindingQueue to use MEF. Refactored usages to run against interface.

* 3278 Corrected ServiceHost namespace. Removed unused dependency to sqlToolsContext in LanguageService. Updated dependency in MetadataService and LanguageService to IProtocolEndpoint instead of ServiceHost

* 3278 Added back NextResult function and summary to KustoResultsReader. Renamed instance to _instance in ConnectionService and DatabaseLocksManager to stay consistent. Changed OpenDataSourceConnection to private in ConnectionService

* 3278 Converted helper methods back to static and removed backing interfaces

* 3278 Reverted AdminService > InitializeService to use ServiceHost as param
2020-08-24 13:18:00 -07:00
Aasim Khan
309e1906ba Changing InvalidOwnerUri string to a more generic one (#1046)
* Change invalid owner uri error message to a generic one

* Adding strings generated by SRGEN step
2020-08-18 09:08:31 -07:00
Lucy Zhang
6c24f72f48 added task to clear diagnostics when file closed (#1050)
* added task to clear diagnostics when file closed

* remove unnecessary return statement
2020-08-17 08:04:09 -07:00
Udeesha Gautam
616a79c83d GetConnectionString API fix to not change the cache only return the value (#1040)
* Get connection string call was changing the connection info (ref object) received from cache. Changing it to just get and make changes to only the returned string..

* Change to ensure PR validation works
2020-08-09 19:17:16 -07:00
Alex Ma
81b4bb7753 Warning for multiple delete (#931)
* Fix for multiple delete bug

* minor space removal

* removed catch error in RowDelete

* small optimizations

* space adding

* WIP on creating test

* Some cleanup

* removed spaces

* Fix for verifytext

* Added check for command format correctness.

* tidying up

* added working test for command exception

* simplification of TestDbDataReader getint

* Corrections made

* spacing and naming issues

* minor space

* one more space issue
2020-08-06 13:55:49 -07:00
David Shiflet
839acf67cd Convert most tools service tests to nunit (#1037)
* Remove xunit dependency from testdriver

* swap expected/actual as needed

* Convert Test.Common to nunit

* port hosting unit tests to nunit

* port batchparser integration tests to nunit

* port testdriver.tests to nunit

* fix target to copy dependency

* port servicelayer unittests to nunit

* more unit test fixes

* port integration tests to nunit

* fix test method type

* try using latest windows build for PRs

* reduce test memory use
2020-08-05 13:43:14 -04:00
Charles Gagnon
bf4911795f Add Notebook conversion logic (#1027)
* Add logic to convert Notebook doc into SQL query

* move classes

* Add logic for SQL -> Notebook

* Fix a few trimming issues
2020-08-03 07:54:06 -07:00
David Shiflet
79e81a69b2 Add an ADO yaml file for integration tests (#1035)
* enable code coverage for ADO test run

* fix debugtype default

* use file name instead of guid

* use wildcard for test dll name
2020-08-02 15:05:17 -04:00
Charles Gagnon
9fbcadb46e Add service for converting a notebook to sql and back (#1022) 2020-07-29 16:31:51 -07:00
Kim Santiago
788dc31081 Update DacFx deploy and generate script with options (#998)
* update deploy and generate script to accept deployment options

* add tests

* add test with option set to true
2020-07-29 13:43:38 -07:00
Kim Santiago
1a93404e54 Add support for getting DacFx deploy options from a publish profile (#995)
* add support for getting options from a publish profile

* update comments

* set values for default options if they aren't specified in the publish profile

* addressing comments
2020-07-27 17:52:15 -07:00
Charles Gagnon
aa56ec852d Fix Null Pointer exception in test (#999)
* Full NPE

* force trigger
2020-07-24 10:05:14 -07:00
Charles Gagnon
9fba9b1955 Add developers startup arg (#997) 2020-07-23 16:09:58 -07:00
Soheil Alizadeh
31659e1126 Prevent ArgumentNullException in refresh request (#963) 2020-06-14 23:24:31 -07:00
Soheil Alizadeh
887a499d2b Update Infrastructure (#961)
* Update form .NET Core 2.2 to .NET Core 3.1

- Global variable for projects
- Change TFMs from netcoreapp2.2 to netcoreapp3.1
- Update global.json
- Update build.json
- Remove direct framework cake TestCore task
- Update travis dotnet version
- Update azure pipline file
- Update vscode launch.json

* Add Central Package Management

* Fix xUnit Breaking Change for MemberData type

* Fix xUnit breaking change for duplicate test method name

* Fix Rang/Index type conflict with System.Rang/Index

* Update vscode tasks.json

* Change serviceHostExecutable path in ServiceTestDriver.cs

* Downgrade SDK version (https://github.com/appveyor/ci/issues/3440)

- Appveyor hasn't installed latest SDK therefore I downgrade it until they install it.

* Dump Microsoft.SqlServer.DACFx
2020-06-12 11:44:18 -07:00
Benjin Dubishar
c51a59dadf Adding SqlCmdVars support for DacFx Deploy (#971)
* Adding SqlCmdVars support for DacFx Deploy

* Adding SqlCmdVars support for GenerateDeployScript

* Consolidating test logic
2020-06-10 16:00:59 -07:00
Karl Burtram
e8e699dbd8 Revert "-Removed forced master condition (#957)" (#972)
This reverts commit 69b93a05f2.
2020-06-09 13:42:29 -07:00
Aleksei Guzev
ca5cbf76e6 Fix TargetName for assessment items (#968) (#969)
* Remove the second appearance of server name in TargetName property
* Replace slash with colon as database name separator in TargetName to avoid ambiguity
Fixes #968
2020-06-08 13:26:25 -07:00
Kim Santiago
836687a41d update schema compare default option (#967) 2020-06-08 11:26:00 -07:00
Satyem-MS
8d6a48a23f Auto Completion for date_bucket (#933) 2020-05-18 11:27:11 -07:00
Aasim Khan
69b93a05f2 -Removed forced master condition (#957) 2020-05-15 10:09:33 -07:00
Udeesha Gautam
9eec757ad1 Updating DacFx nuget to one with SqlTasks (#958)
* Updating DacFx nuget to one with SqlTasks

* Adding common references to build props to avoid multiple places for update
2020-05-08 17:43:04 -07:00
Jeff Trimmer
82eed06847 Add Always Encrypted Parameterization Functionality (#953) 2020-05-05 12:01:24 -07:00
Amir Omidi
e3f1789f18 Enable arm builds (#955) 2020-05-01 15:17:27 -07:00
Alan Ren
8f6662b019 extend the list databases request (#951)
* extend the list databases request

* reuse databaseInfo and refactor

* fix typo
2020-04-28 16:12:56 -07:00
Sakshi Sharma
96df91c8fa Sql Proj Extract from database to Sql files (#949)
* SqlToolsServices changes for  Sql Proj Extract from database to Sql files

* Bumped DacFx version. Addressed comments.
2020-04-27 18:35:19 -07:00
Kim Santiago
314627f83c Show file location for dacfx extract and export (#936)
* show file location for dacfx extract and export

* Add TargetDestination field to TaskInfo

* remove setting database and server name to null if there's packageFilePath

* remove unnecessary parameter
2020-04-27 17:37:08 -07:00
Aleksei Guzev
bcc1f2a486 Add AQL Assessment service (#946)
[SQL Assessment API](https://docs.microsoft.com/en-us/sql/sql-assessment-api/sql-assessment-api-overview) provides a mechanism to evaluate the configuration
of SQL Server for best practices. SQL Assessment API gives a list
of recommended actions to improve SQL Server performance or security.

The SQL Assessment service is used by the expected SQL Assessment
feature of Azure Data Studio. 

SqlAssessmentService forwards JSONRPC calls to SQL Assessment engine
and wraps results as a response.

`assessment/getAssessmentItems` returns a set of checks 
applicable to a given target.

`assessment/invoke` returns a set of recommendations
for improving SQL Server instance or database configurations. 

`assessment/generateScript` returns a T-SQL script for storing
an assessment result set to a SQL data table.
2020-04-24 10:52:55 +03:00
Kim Santiago
bf6c573e7b bump nuget package (#942) 2020-04-07 16:25:57 -07:00
Charles Gagnon
7d37de218a Fix to not add parens for global variable built in functions (#940) 2020-04-07 09:56:24 -07:00