* Renames ShowPlan directories along with corresponding namespaces
* Renames ShowPlanGraphUtils to ExecutionPlanGraphUtils
* Revert "Renames ShowPlanGraphUtils to ExecutionPlanGraphUtils"
This reverts commit 5dc2696ae906598447eed7360a3f342218432b83.
* Reverts show plan tests name change.
* Renames show plan test XML files.
* Renames ported directory to ShowPlan and updates namespace accordingly
* Initial Investigation
* Working code with include, connect, on error and tests
* Adding some loc strings
* Some cleanup and more tests
* Some dummy change to trigger build
* Adding PR comments
* Addressing PR comments
* Revert "Update SMO (#875)"
This reverts commit 96593d66e3.
* Revert "Switch Tools Service to new SMO and Microsoft.Data.SqlClient driver (#865)"
This reverts commit 9d140b53f3.
* Bump SMO to 160.1910315.0-preview
* switch to ambient props and targets files
* build against Microsoft.Data.SqlClient
* build tests
* fix test bug
* temporarily add SMO nuget to the repo
* update to released Microsoft.Data package
This changes adds the following two notifications from the results processing within a batch. These new notifications allows a consumer to stream results from a resultset instead of getting them all at once after the entire resultset has been fetched.
ResultsAvailable
This is issued after at least 1 row has been fetched for this resultset.
ResultsUpdated
This is issued periodically as more rows are available on this resultset. The final send of this notification when all rows have been fetched has the property 'Complete' set to true in the ResultSummary object.
Detailed Change Log:
* Initial completed implementation of QueryResults stream feature. 3 unittests still need fixing
* Fix for the 3 failing test. I will look into making MockBehavior strict again for the three tests later
* Making GetReader/GetWriter use filestream objects in FileShare.ReadWrite mode so the file can be concurrently read and written
* Changing resultsAvailable also to fire off on a timer instead of after 1st row
* adding a project for clr TableValuedFunction to produce result set with delays after each row. This is helpful in end to end testing.
* Fixing up some tests and simplifying implementation of result update timer
* Address review comments
* Some test fixes
* Disabled flaky test verification
* Reopen connections prior to creating query execution data readers
* Reopen connection if an exception was rasied executing the query
* Fix unit tests
This change modifies the logging framework within sqltoolservice.
Moves away from custom Logger object to start using .Net tracing framework. It supports for the static Trace and TraceSource way of logging. For all new code it is recommend that we log the log messages using the existing static Logger class, while the code changes will continue to route the older Trace.Write* calls from the process to same log listeners (and thus the log targets) as used by the Logger class. Thus tracing in SMO code that uses Trace.Write* methods gets routed to the same file as the messages from rest of SQLTools Service code.
Make changes to start using .Net Frameworks codebase for all logging to unify our logging story.
Allows parameter to set tracingLevel filters that controls what kinds of message make it to the log file.
Allows a parameter to set a specific log file name so if these gets set by external code (the UI code using the tools service for example) then the external code is aware of the current log file in use.
Adding unittests to test out the existing and improved logging capabilities.
Sequences of checkins in development branch:
* Saving v1 of logging to prepare for code review. Minor cleanup and some end to end testing still remains
* Removing local launchSettings.json files
* added support for lazy listener to sqltoolsloglistener and removed incorrect changes to comments across files in previous checkin
* Converting time to local time when writing entries to the log
* move the hosting.v2 to new .net based logging code
* removing *.dgml files and addding them to .gitignore
* fixing typo of defaultTraceSource
* Addressing pull request feedback
* Adding a test to verify logging from SMO codebase
* propogating changes to v1 sqltools.hosting commandoptions.cs to the v2 version
* Fixing comments on start and stop callstack methods and whitespaces
* Commenting a test that got uncommented by mistake
* addding .gitattributes file as .sql file was observed to be misconstrued as a binary file
* fixed bug where execution time was wrong for multiple batches
* fixed bug where multiple execution would show wrong execution time
* simplified logic
* Support `GO N` syntax to execute multiple times
- Plumbed through the batch execution count from the parser and used in the batch execution code path
- Functionality matches SSMS:
- Outputs loop start/end messages that match SSMS if you're doing multi-batch execution
- Outputs an "ignoring failure" error if an error happens during a batch
- Added tests for this
- Manually verified end to end also
* Fixing test error
* Changing query/subset API to only use Result on success, Error on error
* Creating an interservice API for getting query result subsets
* Updates to subset API
* RowStartIndex is now long
* Output of query/subset is a 2D array of DbCellValue
* Adding LongSkip method to LongList to allow skipping ahead by longs
* Moving LongList back to ServiceLayer utilities. Move refactoring
* Stubbing out request for edit/subset
* Initial implementation of getting edit rows
* Unit tests for RowEdit and RowDelete .GetEditRow
* Fixing major bugs in LongList implementation, adding much more thorough tests
* Adding some more unit tests and fixes to make unit tests pass
* Fixing comment
The main goal of this feature is to enable a command that will
1) Generate a parameterized command for each edit that is in the session
2) Execute that command against the server
3) Update the cached results of the table/view that's being edited with the committed changes (including computed/identity columns)
There's some secret sauce in here where I cheated around worrying about gaps in the updated results. This was accomplished by implementing an IComparable for row edit objects that ensures deletes are the *last* actions to occur and that they occur from the bottom of the list up (highest row ID to lowest). Thus, all other actions that are dependent on the row ID are performed first, then the largest row ID is deleted, then next largest, etc. Nevertheless, by the end of a commit the associated ResultSet is still the source of truth. It is expected that the results grid will need updating once changes are committed.
Also worth noting, although this pull request supports a "many edits, one commit" approach, it will work just fine for a "one edit, one commit" approach.
* WIP
* Adding basic commit support. Deletions work!
* Nailing down the commit logic, insert commits work!
* Updates work!
* Fixing bug in DbColumnWrapper IsReadOnly setting
* Comments
* ResultSet unit tests, fixing issue with seeking in mock writers
* Unit tests for RowCreate commands
* Unit tests for RowDelete
* RowUpdate unit tests
* Session and edit base tests
* Fixing broken unit tests
* Moving constants to constants file
* Addressing code review feedback
* Fixes from merge issues, string consts
* Removing ad-hoc code
* fixing as per @abist requests
* Fixing a couple more issues
Adding a new setting to query execution setting that will change the display value we generate for `BIT` columns. The new setting is `DefaultDisplayBitAsNumber`. If true, bit columns will be displayed as 1 or 0. If false, they'll be displayed as true or false. The default value is true, to keep parity with SSMS behavior.
Enables us to solve https://github.com/Microsoft/vscode-mssql/issues/690 and https://github.com/Microsoft/vscode-mssql/issues/513
* experimental showplan implementation (tools side only)
* fix for redundant massages from showplan executions
* moved showplan batches to new variables to make it less confusing
* returns showplan as part of batch summary with in each result summary
* cleaned up showplan resultsets
* cleaning up code and making showplan var optional
* changes all var names to showplan
* adding estimated support
* small fixes
* updatin var names and adding EPOptions struct
* adding ssms execution plan logic based on server types
* adding special actions logic
* removing redundant name changes
* execution plan query handler added
* cleaning up functions and data structures
* seperated special actions into its own class
* cleaning up special actions
* cleaning up code
* small new line fixes
* commenting out pre-yukon code
* removing all pre yukon code
* last yukon code commented out
* fixes broken tests
* adding related unit tests; integration tests incoming
* finishing tests and cleaning up code
* semantic changes
* cleaning up semantics
* changes and test fixes, also adding new exceptions into RS
* fixing special actions and cleaning up request logic
* fixing comment to trigger new build
* triggering another build
* fixed up specialaction and added tests for it
This change is a reworking of the way that messages are sent to clients from the service layer. It is also a reworking of the protocol to ensure that all formulations of query send back events to the client in a deterministic ordering. To support the first change:
* Added a new event that will be sent when a message is generated
* Messages now indicate which Batch (if any) generated them
* Messages now indicate if they were error level
* Removed message storage in Batch objects and BatchSummary objects
* Batch objects no longer have error state
It's an overhaul of the Save As mechanism to utilize the file reader/writer classes to better align with the patterns laid out by the rest of the query execution. Why make this change? This change makes our code base more uniform and adherent to the patterns/paradigms we've set up. This change also helps with the encapsulation of the classes to "separate the concerns" of each component of the save as function.
* Replumbing the save as execution to pass the call down the query stack as QueryExecutionService->Query->Batch->ResultSet
* Each layer performs it's own parameter checking
* QueryExecutionService checks if the query exists
* Query checks if the batch exists
* Batch checks if the result set exists
* ResultSet checks if the row counts are valid and if the result set has been executed
* Success/Failure delegates are passed down the chain as well
* Determination of whether a save request is a "selection" moved to the SaveResultsRequest class to eliminate duplication of code and creation of utility classes
* Making the IFileStream* classes more generic
* Removing the requirements of max characters to store from the GetWriter method, and moving it into the constructor for the temporary buffer writer - the values have been moved to the settings and given defaults
* Removing the individual type writers from IFileStreamWriter
* Removing the individual type writers from IFIleStreamReader
* Adding a new overload for WriteRow to IFileStreamWriter that will write out data, given a row's worth of data and the list of columns
* Creating a new IFileStreamFactory that creates a reader/writer pair for reading from the temporary files and writing to CSV files
* Creating a new IFileStreamFactory that creates a reader/writer pair for reading from the temporary files and writing to JSON files
* Dramatically simplified the CSV encoding functionality
* Removed duplicated logic for saving in different types and condensed down to a single chain that only differs based on what type of factory is provided
* Removing the logic for managing the list of save as tasks, since the ResultSet now performs the actual saving work, there's no real need to expose the internals of the ResultSet
* Adding new strings to the sr.strings file for save as error messages
* Completely rewriting the unit tests for the save as mechanism. Very fine grained unit tests now that should cover majority of cases (aside from race conditions)
* Refactoring maxchars params into settings and out of file stream factory
* Removing write*/read* methods from file stream readers/writers
* Migrating the CSV save as to the resultset
* Tweaks to unit testing to eliminate writing files to disk
* WIP, moving to a base class for save results writers
* Everything is wired up and compiles
* Adding unit tests for CSV encoding
* Adding unit tests for CSV and Json writers
* Adding tests to the result set for saving
* Refactor to throw exceptions on errors instead of calling failure handler
* Unit tests for batch/query argument in range
* Unit tests
* Adding service integration unit tests
* Final polish, copyright notices, etc
* Adding NULL logic
* Fixing issue of unicode to utf8
* Fixing issues as per @kburtram code review comments
* Adding files that got broken?
* DbColumn and ReliableConnection tests
* More retry connection tests
* More tests
* Fix broken peek definition integration tests
* Fix test bug
* Add a couple batch tests
* Add some more tests
* More tests for code coverage.
* Validation and Diagnostic tests
* A few more tests
* A few mote test changes.
* Update file path tests to run on Windows only
This change is part of the progressive results code. It will submit a notification from the service layer to indicate when execution of a batch has completed. This notification will contain the selection for batch, execution start time, and its ID. This will enable the extension to produce a header for the batch before the batch completes, in order to make it more clear to the user that execution is going on.
* Adding new event for batch start
* Unit tests
* Fixing comments as per @kevcunnane
The main change in this pull request is to add a new event that will be fired upon completion of a resultset but before the completion of a batch. This event will only fire if a resultset is available and generated.
Changes:
* ConnectionService - Slight changes to enable mocking, cleanup
* Batch - Moving summary generation into ResultSet class, adding generation of ordinals for resultset and locking of result set list (which needs further refinement, but would be outside scope of this change)
* Adding new event and associated parameters for completion of a resultset. Params return the resultset summary
* Adding logic for assigning the event a handler in the query execution service
* Adding unit tests for testing the new event /making sure the existing tests work
* Refactoring some private properties into member variables
* Refactor to remove SectionData class in favor of BufferRange
* Adding callback for batch completion that will let the extension know that a batch has completed execution
* Refactoring to make progressive results work as per async query execution
* Allowing retrieval of batch results while query is in progress
* reverting global.json, whoops
* Adding a few missing comments, and fixing a couple code style bugs
* Using SelectionData everywhere again
* One more missing comment
* Adding new notification type for result set completion
* Plumbing event for result set completion
* Unit tests for result set events
This includes a fairly substantial change to create a mock of the
ConnectionService and to create separate memorystream storage arrays. It
preserves more correct behavior with a integration test, fixes an issue
where the test db reader will return n-1 rows because the Reliable
Connection Helper steals a record.
* Adding locking to ResultSets for thread safety
* Adding/fixing unit tests
* Adding batch ID to result set summary
The main feature of this pull request is a new callback that's added to the query class that is called when a batch has completed execution and retrieval of results. This callback will send an event to the extension with the batch summary information. After that, the extension can submit subset requests for the resultsets of the batch.
Other smaller changes in this pull request:
Refactor to assign a batch a id when its created instead of when returning the list of batch summaries
Passing the SelectionData around instead of extracting the values for it
Moving creation of BatchSummary into the Batch class
Retrieval of results is now permitted even if the entire query has not completed, as long as the batch requested has completed.
Also note, this does not break the protocol. It adds a new event that a queryRunner can listen to, but it doesn't require it to be listened to.
* Refactor to remove SectionData class in favor of BufferRange
* Adding callback for batch completion that will let the extension know that a batch has completed execution
* Refactoring to make progressive results work as per async query execution
* Allowing retrieval of batch results while query is in progress
* reverting global.json, whoops
* Adding a few missing comments, and fixing a couple code style bugs
* Using SelectionData everywhere again
* One more missing comment
* Make save results asynchronous
* Prevent write share of file
* Lock objects in stages
* Create Save result objects
* refactor and write rows in batches
* CHange batchSize from test value
* Remove await in handler
* Removing the file reader as a member of the resultset
* Change Dispose to wait for save
* Change concurrentBag
* PascalCase variables
* Modify function signature and tests
* Safe file methods
* refactor ResultSets to Ilist and remove ToList
* Change dictionary key and prevent add to saveTasks during dispose
* Simplify row concatenation
* Fix prevent add
* Fix prevent add
* Add methods to expose saveTasks and isBeingDisposed
After much thinking, this change brings the message behavior into line with SSMS, including if the NOCOUNT is set. This is a somewhat non-obvious solution, but the StatementCompleted event handler will only be fired if there is a record count to return. We'll add the message for number of records if the StatementCompleted event is fired, otherwise we won't add any messages while processing the resultsets of the batch. If any messages are returned from the server, we'll capture those. Then, if at the end of the batch, we haven't collected any messages from StatementCompleted events or server messages, then we'll add the "completed successfully" message.
This matches behavior of SSMS that will only emit a "completed successfully" message if there were no other messages for the batch.
* Solution to issue, some unit tests needed to be tweaked
* Comments for the event handler
Moving some logic around such that when a query is cancelled, it isn't thrown away, allowing reading of the partial results, a la SSMS.
Adding a configure await to fix a tenacious bug causing query cancellations to hang for 20s or more.
Capturing sql errors for user cancellation, to return all user cancelation scenarios using the same messages. (ie, cancelling during ExecuteReaderAsync will yield a error from the server whereas cancelling during ReadAsync throws a TaskCancelledException
No changes to protocol, just implementation changes.
* Test of try/finally
* Fixed issue where resultsets are unreadable after query cancellation
* Fix for await/async issue
The two main changes in this pull request:
Launching query execution as an asynchronous task that performs a callback upon completion or failure of a query. (Which also sets us up for callbacks progressive results)
Moving away from using the Result of a query execution to return an error. Instead we'll use an error event to return an error
Additionally, some nice refactoring and cleaning up of the unit tests to take advantage of the cool RequestContext mock tooling by @kevcunnane
* Initial commit of refactor to run execution truely asynchronously
* Moving the storage of the task into Query class
Callbacks for completion of a query and failure of a query are setup as
events in the Query class. This actually sets us up for a very nice
framework for adding batch and resultset completion callbacks.
However, this also exposes a problem with cancelling queries and returning
errors -- we don't properly handle errors during execution of a query
(aside from DB errors).
* Wrapping things up in order to submit for code review
* Adding fixes as per comments
* added support for timestamps
* fixed tests
* Moved message class to own file; added 'z' to end of date strings
* added default time constructor
* removed unnecessary z
* added time string format info in comment
* changed from utc time to using local time
* Do not use ReliableCommand in the query execution service.
* Fixing the logic to remove InfoMessage handlers from ReliableSqlConnection
* Adding test to query UDT
* inital pipe of line numbers and getting text from workspace services
* tests compile
* Fixed bug regarding tests using connections on mac
* updated tests
* fixed workspace service and fixed tests
* integrated feedback
* Strings sweep for connection service
* String sweep for credentials service
* String sweep for hosting
* String sweep for query execution service
* String sweep for Workspace service
* Renaming utility namespace to match standards
Renaming Microsoft.SqlTools.EditorServices.Utility to
Microsoft.SqlTools.ServiceLayer.Utility to match the naming changes done a
while back. Also renaming them on the files that use them
* Namespace change on reliable connection
* Adding the new resx and designer files
* Final bug fixes for srgen
Fixing flakey moq package name
* Removing todo as per @kevcunnane
* Adding using statements as per @llali's comment
* Fixing issues from broken unit tests
Note: This feature contains changes that will break the contract for
saving as CSV and JSON. On success, null is returned as a message instead
of "Success". Changes will be made to the vscode component to handle this
change.
