* Adding file writer for Markdown tables. No testing yet.
* Unit tests for the markdown writer
* Wiring up the factory and and request types
* Wiring up changes for Markdown serialization in serialization service
* Couple last minute tweaks
* Changes as per PR comments
* Revert temp testing code. 🙈
* Fluent assertions in SerializationServiceTests.cs
Co-authored-by: Ben Russell <russellben@microsoft.com>
* Minimal changes to make headers appear on empty result sets
* Columns for everyone!
* Updating tests - some don't pass yet
* Adding some more tests to verify the changes for column/row selection
* null default columns
* Updates to comments as per PR comments
* Moving logic for adding default values to new rows
* Fixing implementation of script generation to handle default values all around
* Unit tests!
* WIP
* Reworking row create script/command generation to work more cleanly and work on triggered tables
* Addressing some bugs with the create row implementation
* Implementing the trigger table fix for row updates
Some small improvements to the create/update tests.
* Refactoring sql script formatting helpers into To and From helpers
* Updates to make error messages for formatting errors more useful
* Fixing dumb breaks in unit tests
* Addressing comments from PR
* Updates to the SR files...
* WIP
* All the new RowCreate tests are working
* Fixing a couple bugs with the row delete and row update tests
* Regenerating localization files
* Fixing multiple iteration in tests
* Moving logic for adding default values to new rows
* Fixing implementation of script generation to handle default values all around
* Unit tests!
* Addressing PR comments
* Fix to make sql exceptions surface properly to user (with important notes!)
* Adding support for detecting column size issues when updating a cell
* Adding unit tests for the exception on read scenario
* Replacing Dictionary with ConcurrentDictionary since values are accessed in async contexts
* Adding new method to allow async tasks to be executed in the exception continuation
* Adding unit tests for the aforementioned
* Adding exception handling to async tasks in file browser service
* Updating query execution async handling to use the async version
* Removing unnecesary send result from continuewithonfaulted
* WIP
* This code makes it work!
* Adding similar exception hanling behavior to saving result sets
* Adding unit tests for new extension methods
Auto-cleanup of proj file whitespace
* Implementing changes as per code review comments
* Adding unit tests for save as excel
* Adding unit test for revert cell integration
* Adding unit test for edit session initialization
* Fixing issue where excel file factory wasn't being overridden
* Fixing issue where Guids weren't being stored in the service buffer file as Guids
* Removing unnecessary ReadSqlGuid method
* Adding unit test for revert cell integration
* Adding unit test for edit session initialization
* Revert "Adding unit test for revert cell integration"
This reverts commit a949926f2ebbb0c39f776edba76d999af4f3f3e9.
* Revert "Adding unit test for edit session initialization"
This reverts commit ff9e5a7d0e5392b27257e57db64879699d73d21c.
Update to include the Row ID with a DbCellValue. This will be super useful to us for the purposes of managing individual rows via slick grid.
Although this changes the API for edit/subset and query/subset, it is effectively backwards compatible since it's adding a parameter, not removing anything.
* New DbCellValue has an internal Row ID
* Adding unit tests
This change ensures that when calling `requestContext.SendError` you are only able to supply parameters that match the language service beta protocol expected Error object. In other words, you have to provide an error message and optionally and error code.
# **BREAKING API CHANGES**
This will break displaying errors in Microsoft/vscode-mssql. I will be making changes to properly handle the error object shortly.
* Adding contract for returning Error objects as per LanguageService "protocol"
* Fixes throughout codebase to send only error message in error cases
Cleanup of CredentialServiceTest unit test class
Adding standard error handling for event flow validator
* Adding optional data field as per protocol spec
https://github.com/Microsoft/language-server-protocol/blob/master/protocol.md
* Adding optional validation for error objects
Small change that removes logging complete JSON objects that are sent back from the tools service via JSON RPC. Now the message type and method is logged only. In previous releases, if debug logging was enabled, all JSON objects sent from the tools service would be logged in the debug log, if enabled. This was bad for 2 reasons:
1) sensitive data (passwords from credential service, cell contents) would be logged
2) the log would quickly become massive due to all rows being written to the log, every time they were requested.
This change enhances the way that edit/updateCell and edit/revertCell operations are performed.
## **THE API BREAKING CHANGES**:
* edit/updateCell now returns an EditCell (a DbCellValue with a dirty flag) and a row dirty flag.
* edit/revertCell now returns an EditCell (a DbCellValue with a dirty flag) and a row dirty flag.
If by setting the value of a cell via edit/updateCell the row no longer has any edits (an "implicit revert"), the entire row's edit will be removed from the cache. Additionally, if by requesting edit/revert all the pending edits for a row are removed, the entire row's edit will be removed from the cache. This will prevent issues where committing will generate an invalid script because it has no pending changes.
* Adding EditCell class
Returning EditCell with EditUpdateCellResult
* Adding code that will remove a row update if the row is clean after a cell update
* Adding code that will return an EditCell and row dirty flag when a cell is reverted.
If the row is reverted by the cell revert, the pending update will be removed
* Comments for edit cell
* Changes as per pull request comments
This change adds a couple things
_Multipart Identifier Decoding_
The ability to decode a multipart identifier (with or without escaping) has been added to the SqlScriptFormatter utility class. This code is utilized to split a table name provided to the edit/initialize request into schema and table name.
_Default Schema Workaround_
The code that retrieves the SMO metadata objects originally used the `[]` operator to access the objects. Due to a bug(?) in SMO, this results in problems when loading tables without a default schema (in our case if you're logged in as SA). Using the metadata object constructors gets around this issue, we are explicitly using them.
* Adding decoding of multipart identifiers
Adding code fix for default schema issue
* Adding some more localizable strings for errors when loading metadata
* Adding localization files... again?
* Changes as per pull request comments
* Ensures that metadata is "extended" before creating a new row edit object
* Adds the commit handler to edit data service initialization calls
* Unit tests for the associated changes
* Major refactor of EditDataMetadata providers
* EditMetadataFactory generates "basic" EditTableMetadata objects based entirely on SMO metadata
* SmoEditTableMetadata no longer depends on SMO, making it unecessary to mock it
* Renamed SmoEditTableMetadata to EditTableMetadata
* EditTableMetadata can be extended with DbColumnWrappers
* Moving logic for extending a EditColumnMetadata into that class
* I *think* this will work for async execution of initialize tasks
* Fixing unit tests for new Edit(Table|Column)Metadata classes
* Async stuff that works! And passes unit tests
* Adding unit tests
Adding .idea to gitignore
* Adding message to the EditSessionReadyEvent
* Fixes from dev merge
* Fixing unit tests that Rider didn't catch as failing
May have been a bit heavy-handed with the async/await stuff
* Couple changes as per PR comments
* Changing query/subset API to only use Result on success, Error on error
* Creating an interservice API for getting query result subsets
* Updates to subset API
* RowStartIndex is now long
* Output of query/subset is a 2D array of DbCellValue
* Adding LongSkip method to LongList to allow skipping ahead by longs
* Moving LongList back to ServiceLayer utilities. Move refactoring
* Stubbing out request for edit/subset
* Initial implementation of getting edit rows
* Unit tests for RowEdit and RowDelete .GetEditRow
* Fixing major bugs in LongList implementation, adding much more thorough tests
* Adding some more unit tests and fixes to make unit tests pass
* Fixing comment
Very small feature add to make sure that values entered for `TIME` columns are < 24hrs. If the value is >=24hrs when setting the column, an exception will be thrown that the edit/updateCell handler will catch and convert into an error on the JSON RPC side.
* Adding validation of timespan columns
* Fixing a merge conflict
// edit/dispose -------------------------------------------------------------------------------
export interface EditDisposeParams extends IEditSessionOperationParams { }
export interface EditDisposeResult { }
* Initial plumbing for edit/revertCell
* Implementation of revert cell in the parents of the row edit base
* Adding unit tests
Returns strings for the default value of a column when a new row is created. The values that could come back:
* `null` when there isn't a default for the column
* a string when there is a default for the column
* a placeholder (currently <TBD>) when the column cannot be updated
* Renaming EditTableMetadata to reflect its SMO source
* Implementation of returning default values
* Unit test for default values
* Reworking column defaults using default constraints
* Adding unit test for new sql script unwrapper
* Disabling flaky test
* Fixing oddities in tests, removing personal tests
* Fixing broken unit test
The main goal of this feature is to enable a command that will
1) Generate a parameterized command for each edit that is in the session
2) Execute that command against the server
3) Update the cached results of the table/view that's being edited with the committed changes (including computed/identity columns)
There's some secret sauce in here where I cheated around worrying about gaps in the updated results. This was accomplished by implementing an IComparable for row edit objects that ensures deletes are the *last* actions to occur and that they occur from the bottom of the list up (highest row ID to lowest). Thus, all other actions that are dependent on the row ID are performed first, then the largest row ID is deleted, then next largest, etc. Nevertheless, by the end of a commit the associated ResultSet is still the source of truth. It is expected that the results grid will need updating once changes are committed.
Also worth noting, although this pull request supports a "many edits, one commit" approach, it will work just fine for a "one edit, one commit" approach.
* WIP
* Adding basic commit support. Deletions work!
* Nailing down the commit logic, insert commits work!
* Updates work!
* Fixing bug in DbColumnWrapper IsReadOnly setting
* Comments
* ResultSet unit tests, fixing issue with seeking in mock writers
* Unit tests for RowCreate commands
* Unit tests for RowDelete
* RowUpdate unit tests
* Session and edit base tests
* Fixing broken unit tests
* Moving constants to constants file
* Addressing code review feedback
* Fixes from merge issues, string consts
* Removing ad-hoc code
* fixing as per @abist requests
* Fixing a couple more issues
The goal of this make sure that test code is correctly organized to ensure that test suites aren't dependent on each other.
* UnitTests get their own project now (renaming Microsoft.SqlTools.ServiceLayer.Test to Microsoft.SqlTools.ServiceLayer.UnitTests) which is about 90% of the changes to the files.
* IntegrationTests no longer depends on UnitTests, only Test.Common
* Any shared components from TestObjects that spins up a "live" connection has been moved to IntegrationTests Utility/LiveConnectionHelper.cs
* The dictionary-based mock file stream factory has been moved to Test.Common since it is used by UnitTests and IntegrationTests
* Added a overload that doesn't take a dictionary for when we don't care about monitoring the storage (about 90% of the time)
* The RunIf* wrapper methods have been moved to Test.Common
* OwnerUri and StandardQuery constants have been moved to Test.Common Constants file
* Updating to latest SDK version available at https://www.microsoft.com/net/core#windowscmd
* Moving unit tests to unit test folder
* Changing namespaces to UnitTests
* Moving some constants and shared functionality into common project, making the UnitTests reference it
* Unit tests are working!
* Integration tests are working
* Updating automated test runs
* Fixing one last broken unit test
* Exposing internals for other projects
* Moving edit data tests to UnitTest project
* Applying refactor fixes to unit tests
* Fixing flaky test that wasn't awaiting completion
* Updating to latest SDK version available at https://www.microsoft.com/net/core#windowscmd
* Rolling back to preview2-003156
preview2-1-003177 has a bug where discovery of tests fails with null
exception
preview2-003156 is the latest SDK version on
https://www.microsoft.com/net/download/core (which surprisingly differs
from the version on the front page of .NET Core page)
This is a very large change. I'll try to outline what's going on.
1. This adds the **EditDataService** which manages editing **Sessions**.
1. Each session has a **ResultSet** (from the QueryExecutionService) which has the rows of the table and basic metadata about the columns
2. Each session also has an **IEditTableMetadata** implementation which is derived from SMO metadata which provides more in-depth and trustworthy data about the table than SqlClient alone can.
3. Each session holds a list of **RowEditBase** abstract class implementations
1. **RowUpdate** - Update cells in a row (generates `UPDATE` statement)
2. **RowDelete** - Delete an entire row (generates `DELETE` statement)
3. **RowCreate** - Add a new row (generates `INSERT INTO` statement)
4. Row edits have a collection of **CellUpdates** that hold updates for individual cells (except for RowDelete)
1. Cell updates are generated from text
5. RowEditBase offers some baseline functionality
1. Generation of `WHERE` clauses (which can be parameterized)
2. Validation of whether a column can be updated
2. New API Actions
1. edit/initialize - Queries for the contents of a table/view, builds SMO metadata, sets up a session
2. edit/createRow - Adds a new RowCreate to the Session
3. edit/deleteRow - Adds a new RowDelete to the Session
4. edit/updateCell - Adds a CellUpdate to a RowCreate or RowUpdate in the Session
5. edit/revertRow - Removes a RowCreate, RowDelete, or RowUpdate from the Session
6. edit/script - Generates a script for the changes in the Session and stores to disk
7. edit/dispose - Removes a Session and releases the query
3. Smaller updates (unit test mock improvements, tweaks to query execution service)
**There are more updates planned -- this is just to get eyeballs on the main body of code**
* Initial stubs for edit data service
* Stubbing out update management code
* Adding rudimentary dispose request
* More stubbing out of update row code
* Adding complete edit command contracts, stubbing out request handlers
* Adding basic implementation of get script
* More in progress work to implement base of row edits
* More in progress work to implement base of row edits
* Adding string => object conversion logic and various cleanup
* Adding a formatter for using values in scripts
* Splitting IMessageSender into IEventSender and IRequestSender
* Adding inter-service method for executing queries
* Adding inter-service method for disposing of a query
* Changing edit contract to include the object to edit
* Fully fleshing out edit session initialization
* Generation of delete scripts is working
* Adding scripter for update statements
* Adding scripting functionality for INSERT statements
* Insert, Update, and Delete all working with SMO metadata
* Polishing for SqlScriptFormatter
* Unit tests and reworked byte[] conversion
* Replacing the awful and inflexible Dictionary<string, string>[][] with a much better test data set class
* Fixing syntax error in generated UPDATE statements
* Adding unit tests for RowCreate
* Adding tests for the row edit base class
* Adding row delete tests
* Adding RowUpdate tests, validation for number of key columns
* Adding tests for the unit class
* Adding get script tests for the session
* Service integration tests, except initialization tests
* Service integration tests, except initialization tests
* Adding messages to sr.strings
* Adding messages to sr.strings
* Fixing broken unit tests
* Adding factory pattern for SMO metadata provider
* Copyright and other comments
* Addressing first round of comments
* Refactoring EditDataService to have a single method for handling
session-dependent operations
* Refactoring Edit Data contracts to inherit from a Session and Row
operation params base class
* Copyright additions
* Small tweak to strings
* Updated unit tests to test the refactors
* More revisions as per pull request comments
Adding a new setting to query execution setting that will change the display value we generate for `BIT` columns. The new setting is `DefaultDisplayBitAsNumber`. If true, bit columns will be displayed as 1 or 0. If false, they'll be displayed as true or false. The default value is true, to keep parity with SSMS behavior.
Enables us to solve https://github.com/Microsoft/vscode-mssql/issues/690 and https://github.com/Microsoft/vscode-mssql/issues/513
Fixes a bug where the GetOrOpenConnection method of the ConnectionService would throw if the connection didn't exist. It would yield a very unhelpful "key not found in collection" exception. The code has been updated to be much more helpful and fix the bug.
Adding an OpenAsync method to the ReliableConnection to allow it to properly utilize the cancellation token support of the SqlConnection. It eliminates the need for a "cancellation thread" that has a high likelihood of throwing uncaught exceptions (as per Microsoft/vscode-mssql/issues/547)
* Replacing the awful and inflexible Dictionary<string, string>[][] with a much better test data set class
* Applying all changes
* Removing unused RequestParamTests
Adding new methods for executing queries from other services (such as the upcoming edit data service). The code is written to avoid duplicating logic by using lambdas to perform custom logic.
Additionally, the service host protocol has been slightly modified to split the IMessageSender into IEventSender and IRequestSender. This allows us to use either a ServiceHost or any RequestContext<T> to send events. It becomes very convenient to use another service's request context to send the events for query execution.
**Breaking Change**: This removes the messages property for query dispose results and instead elects to use error for any errors encountered during query disposal. A result is only used when something is successful.
* Splitting IMessageSender into IEventSender and IRequestSender
* Adding inter-service method for executing queries
* Adding inter-service method for disposing of a query
* Adding null checking for the success/error handlers
This is a small API addition that allows us to execute queries directly as strings. This will make it easier to execute queries outside the confines of a workspace like VS Code.
* Refactor execution requests and events are now named less redundantly and moved into a separate namespace for organization. This is the bulk of the changes.
* QueryExecuteBatchNotification -> ExecuteRequests/BatchEvents
* QueryExecuteMessageNotification -> ExecuteRequests/MessageEvent
* QueryExecuteCompleteNotification -> ExecuteRequests/QueryCompleteEvent
* QueryExecuteResultSetCompleteNotification -> ExecuteRequests/ResultSetEvents
* QueryExecuteSubsetRequest -> SubsetRequest.cs
* Creating an inheritance pattern where
* `ExecuteRequestParamsBase` has execution options and ID for a query execute request
* `ExecuteDocumentSelectionParams` inherits from `ExecuteRequestParamsBase` and provides a document selection
* `ExecuteStringParams` inherits from `ExecuteRequestParamsBase` and provides the query text
* Adding a helper method to get SQL text based on request type
* Through the AWESOME POWER OF POLYMORPHISM, we are able to create a request for executing straight SQL basically for free.
* **Breaking change:** query/execute => query/executeDocumentSelection to make it more obvious what is expected.
* Adding unit tests for the code that gets SQL text
* Refactoring of execute contracts into their own namespace
* Refactoring application
* Adding new request for executing queries as strings
* Adding forgotten string request
* Changing the logic for checking the request param types
* Removing redundant declarations
This change is a reworking of the way that messages are sent to clients from the service layer. It is also a reworking of the protocol to ensure that all formulations of query send back events to the client in a deterministic ordering. To support the first change:
* Added a new event that will be sent when a message is generated
* Messages now indicate which Batch (if any) generated them
* Messages now indicate if they were error level
* Removed message storage in Batch objects and BatchSummary objects
* Batch objects no longer have error state
It's an overhaul of the Save As mechanism to utilize the file reader/writer classes to better align with the patterns laid out by the rest of the query execution. Why make this change? This change makes our code base more uniform and adherent to the patterns/paradigms we've set up. This change also helps with the encapsulation of the classes to "separate the concerns" of each component of the save as function.
* Replumbing the save as execution to pass the call down the query stack as QueryExecutionService->Query->Batch->ResultSet
* Each layer performs it's own parameter checking
* QueryExecutionService checks if the query exists
* Query checks if the batch exists
* Batch checks if the result set exists
* ResultSet checks if the row counts are valid and if the result set has been executed
* Success/Failure delegates are passed down the chain as well
* Determination of whether a save request is a "selection" moved to the SaveResultsRequest class to eliminate duplication of code and creation of utility classes
* Making the IFileStream* classes more generic
* Removing the requirements of max characters to store from the GetWriter method, and moving it into the constructor for the temporary buffer writer - the values have been moved to the settings and given defaults
* Removing the individual type writers from IFileStreamWriter
* Removing the individual type writers from IFIleStreamReader
* Adding a new overload for WriteRow to IFileStreamWriter that will write out data, given a row's worth of data and the list of columns
* Creating a new IFileStreamFactory that creates a reader/writer pair for reading from the temporary files and writing to CSV files
* Creating a new IFileStreamFactory that creates a reader/writer pair for reading from the temporary files and writing to JSON files
* Dramatically simplified the CSV encoding functionality
* Removed duplicated logic for saving in different types and condensed down to a single chain that only differs based on what type of factory is provided
* Removing the logic for managing the list of save as tasks, since the ResultSet now performs the actual saving work, there's no real need to expose the internals of the ResultSet
* Adding new strings to the sr.strings file for save as error messages
* Completely rewriting the unit tests for the save as mechanism. Very fine grained unit tests now that should cover majority of cases (aside from race conditions)
* Refactoring maxchars params into settings and out of file stream factory
* Removing write*/read* methods from file stream readers/writers
* Migrating the CSV save as to the resultset
* Tweaks to unit testing to eliminate writing files to disk
* WIP, moving to a base class for save results writers
* Everything is wired up and compiles
* Adding unit tests for CSV encoding
* Adding unit tests for CSV and Json writers
* Adding tests to the result set for saving
* Refactor to throw exceptions on errors instead of calling failure handler
* Unit tests for batch/query argument in range
* Unit tests
* Adding service integration unit tests
* Final polish, copyright notices, etc
* Adding NULL logic
* Fixing issue of unicode to utf8
* Fixing issues as per @kburtram code review comments
* Adding files that got broken?
This is a reworking of the unit tests to permit us to better test events from the service host. This new Event Flow Validator class allows creating a chain of events that can then be validated after execution of the request. Each event can have its own custom validation logic for verifying that the object sent via the service host is correct. It also allows us to validate that the order of events are correct.
The big drawback is that (at this time) the validator cannot support asynchronous events or non-determinant ordering of events. We don't need this for the query execution functionality despite messages being sent asynchronously because async messages aren't sent during unit tests (due to the db message event only being present on SqlDbConnection classes). If the need arises to do async or out of order event validation, then I have some ideas for how we can do that.
* Applying the event flow validator to the query execution service integration tests
* Undoing changes to events that were included in cherry-picked commit
* Cleaning up event flow validation to query execution
* Add efv to cancel tests
* Adding efv to dispose tests
* Adding efv to subset tests
* Adding efv to SaveResults tests
* Copyright
This basically is a replacement to the fix for Adding Milliseconds to DateTime fields. I didn't take into consideration that `DATE` columns would report as DateTime type. Date columns have a numeric scale set to 255, leading to the formatting string for the date time to include 255 millisecond places, which is invalid.
This change also reverses the change to store DateTime precision in the buffer file. Instead, the column metadata is now used when deserializing the rows from the db. `DATETIME` and `DATETIME2` columns are differentiated by their numeric scale while `DATE` columns are differentiated by their datatype name field.
More unit tests were added. Additionally, this fixes an unreported bug that `DATE` columns were being displayed with times, which is incorrect.
* Revert "Adding Milliseconds to DateTime fields (#173)"
This reverts commit 431dfa4156.
* Reworking the reader to use the column metadata for date types
* DbColumn -> DbColumnWrapper
* Fina tweaks to support DATETIME2(0)
* Removing the unused arguments
This change is a refactor of the DbColumnWrapper class. This fixes a huge oversight/gotcha in the DbColumnWrapper where treating the DbColumnWrapper as a DbColumn (ie, polymorphism) would result in almost all the fields of the column being null. This is a highly unexpected behavior. This change fixes that.
This is a slightly larger change than anticipated due the difference between `DATETIME`, `DATETIME2`, and `DateTime`. The `DATETIME` type always uses 3 decimal points of a second, while the `DATETIME2` type has 7 (although since `DATETIME2(7)` is default in SSMS suggesting that it is a variable precision type). Regardless of the db type, the engine returns `DateTime` C# type. The db types are only made visible via the column info, via the numeric precision and numeric scale. My findings were as such:
`DATETIME `: Precision = 23, Scale = 3
`DATETIME2`: Precision = 255, Scale = 7
The scale corresponds neatly with the number of second decimal points to show. The buffer file writer was modified to store both the scale and the number of ticks. Then the buffer file reader was modified to read in the precision and the number of ticks and generate the ToString version of the DateTime to add "f" as many times as there is scale, which corresponds to milliseconds.
* Code for writing milliseconds of datetime/datetime2 columns
* Adding unit tests
* Fixing potential bug with datetime2(0)
This change is part of the progressive results code. It will submit a notification from the service layer to indicate when execution of a batch has completed. This notification will contain the selection for batch, execution start time, and its ID. This will enable the extension to produce a header for the batch before the batch completes, in order to make it more clear to the user that execution is going on.
* Adding new event for batch start
* Unit tests
* Fixing comments as per @kevcunnane
The main change in this pull request is to add a new event that will be fired upon completion of a resultset but before the completion of a batch. This event will only fire if a resultset is available and generated.
Changes:
* ConnectionService - Slight changes to enable mocking, cleanup
* Batch - Moving summary generation into ResultSet class, adding generation of ordinals for resultset and locking of result set list (which needs further refinement, but would be outside scope of this change)
* Adding new event and associated parameters for completion of a resultset. Params return the resultset summary
* Adding logic for assigning the event a handler in the query execution service
* Adding unit tests for testing the new event /making sure the existing tests work
* Refactoring some private properties into member variables
* Refactor to remove SectionData class in favor of BufferRange
* Adding callback for batch completion that will let the extension know that a batch has completed execution
* Refactoring to make progressive results work as per async query execution
* Allowing retrieval of batch results while query is in progress
* reverting global.json, whoops
* Adding a few missing comments, and fixing a couple code style bugs
* Using SelectionData everywhere again
* One more missing comment
* Adding new notification type for result set completion
* Plumbing event for result set completion
* Unit tests for result set events
This includes a fairly substantial change to create a mock of the
ConnectionService and to create separate memorystream storage arrays. It
preserves more correct behavior with a integration test, fixes an issue
where the test db reader will return n-1 rows because the Reliable
Connection Helper steals a record.
* Adding locking to ResultSets for thread safety
* Adding/fixing unit tests
* Adding batch ID to result set summary
This is another large code review. I want to make a few more changes, but since these changes will stand on their own, I'll hold back on making this change set any larger than it already is.
Changes in this request:
To address Microsoft/vscode-mssql#326, instead of doing taskkill on the service layer when WaitForExit is executed, we now make an educated guess at which service layer was spawned when the test starts and do a Process.Kill on it when we shut down the test.
All the perf tests have been moved into a new project. This was done to keep them easily separated from code coverage test runs. At the same time the perf tests were separated into separate classes for logical categorization. This process will likely be repeated on the stress tests. The tests can still easily be ran from Visual Studio Test Explorer
To address Microsoft/vscode-mssql#349, a new SelfCleaningFile class was created to allow for easy cleanup of temporary files generated for integration tests via using blocks.
Due to some of the refactoring done while moving the perf tests to a new project, the TestBase class had to be switched to more of a helper class style. As such, all tests that use inherit from TestBase now create a TestBase object on start via a using block. This also simplifies the cleanup at the end of the test.
* Solution for hanging code coverage runs
Code coverage runs would hang in certain scenarios if a test failed before
the service process could be spawned. The taskkill command would fail to
find the service process. The test would then wait for opencover to exit,
but it would not since the service process it had spawned would still be
running, causing the test run to hang indefinitely.
Solution was to capture the service process after it launched and
explicitly kill it when shutting down the test driver.
* Setting the test name in the propery in the class and removign the parameter from each method
* New project for perf tests
* Reworking integration tests to cleanup temp files
* Changes as per @llali review comments
* Adding copyright notices
* Renaming TestBase => TestHelper
* Renaming SelfCleaningFile => SelfCleaningTempFile
* Removing code that sets TestName property
* Fixing compilation error due to removed code
This is a fairly large set of changes to the unit tests that help isolate the effectiveness of the unit tests.
* Unit tests for query execution have been split into separate files for different classes.
* Unit tests have been added for the ResultSet class which previously did not have tests
* The InMemoryStreamWrapper has been improved to share memory, creating a simulated filesystem
* Creating a mock ConnectionService to decrease noisy exceptions and prevent "row stealing". Unfortunately this lowers code coverage. However, since the tests that touched the connection service were not really testing it, this helps keep us honest. But it will require adding more unit tests for connection service.
* Standardizing the await mechanism for query execution
* Cleaning up the mechanism for getting WorkspaceService mocks and mock FileStreamFactories
* Refactor the query execution tests into their own files
* Removing tests from ExecuteTests.cs that were moved to separate files
* Adding tests for ResultSet class
* Adding test for the FOR XML/JSON component of the resultset class
* Setting up shared storage between file stream readers/writers
* Standardizing on Workspace mocking, awaiting execution completion
* Adding comment for ResultSet class
* Adding useful unit tests for this functionality
* Adding callback functionality for when a file is closed
* Fixing bad data issue w/closing/opening untitled doc
* Adding useful unit tests for this functionality
* Adding callback functionality for when a file is closed
* Moving from public to internal
* Adding useful unit tests for this functionality
* Adding callback functionality for when a file is closed
* Fixing bad data issue w/closing/opening untitled doc
* Adding useful unit tests for this functionality
* Adding callback functionality for when a file is closed
* Moving from public to internal
This will make it much easier for Visual Studio users to validate the TestDriver project.
* Adding TestDriver project and CodeCoverage folder to the main solution
* Adding fallback value for service layer exe path to allow tests to run in Visual Studio
The main feature of this pull request is a new callback that's added to the query class that is called when a batch has completed execution and retrieval of results. This callback will send an event to the extension with the batch summary information. After that, the extension can submit subset requests for the resultsets of the batch.
Other smaller changes in this pull request:
Refactor to assign a batch a id when its created instead of when returning the list of batch summaries
Passing the SelectionData around instead of extracting the values for it
Moving creation of BatchSummary into the Batch class
Retrieval of results is now permitted even if the entire query has not completed, as long as the batch requested has completed.
Also note, this does not break the protocol. It adds a new event that a queryRunner can listen to, but it doesn't require it to be listened to.
* Refactor to remove SectionData class in favor of BufferRange
* Adding callback for batch completion that will let the extension know that a batch has completed execution
* Refactoring to make progressive results work as per async query execution
* Allowing retrieval of batch results while query is in progress
* reverting global.json, whoops
* Adding a few missing comments, and fixing a couple code style bugs
* Using SelectionData everywhere again
* One more missing comment
This isn't much of a change from the change that was backed out. The only change was a version bump of the SRGen nuget package.
* Reenabling running SRGen when running lab builds, improving failure logic, updating SRGen package
* Vbump for srgen package testing
* Final vbump of srgen package
After much thinking, this change brings the message behavior into line with SSMS, including if the NOCOUNT is set. This is a somewhat non-obvious solution, but the StatementCompleted event handler will only be fired if there is a record count to return. We'll add the message for number of records if the StatementCompleted event is fired, otherwise we won't add any messages while processing the resultsets of the batch. If any messages are returned from the server, we'll capture those. Then, if at the end of the batch, we haven't collected any messages from StatementCompleted events or server messages, then we'll add the "completed successfully" message.
This matches behavior of SSMS that will only emit a "completed successfully" message if there were no other messages for the batch.
* Solution to issue, some unit tests needed to be tweaked
* Comments for the event handler
Moving some logic around such that when a query is cancelled, it isn't thrown away, allowing reading of the partial results, a la SSMS.
Adding a configure await to fix a tenacious bug causing query cancellations to hang for 20s or more.
Capturing sql errors for user cancellation, to return all user cancelation scenarios using the same messages. (ie, cancelling during ExecuteReaderAsync will yield a error from the server whereas cancelling during ReadAsync throws a TaskCancelledException
No changes to protocol, just implementation changes.
* Test of try/finally
* Fixed issue where resultsets are unreadable after query cancellation
* Fix for await/async issue
The two main changes in this pull request:
Launching query execution as an asynchronous task that performs a callback upon completion or failure of a query. (Which also sets us up for callbacks progressive results)
Moving away from using the Result of a query execution to return an error. Instead we'll use an error event to return an error
Additionally, some nice refactoring and cleaning up of the unit tests to take advantage of the cool RequestContext mock tooling by @kevcunnane
* Initial commit of refactor to run execution truely asynchronously
* Moving the storage of the task into Query class
Callbacks for completion of a query and failure of a query are setup as
events in the Query class. This actually sets us up for a very nice
framework for adding batch and resultset completion callbacks.
However, this also exposes a problem with cancelling queries and returning
errors -- we don't properly handle errors during execution of a query
(aside from DB errors).
* Wrapping things up in order to submit for code review
* Adding fixes as per comments
Fixing a bug where in various situations, the files used for temporary storage of query results would be leftover. In particular, the following changes were made:
* When the dispose query request is submitted, the corresponding query is now disposed in addition from being removed from the list of active queries
* When a query is cancelled, it is disposed after it is cancelled
* If a query already exists for a given ownerURI, the existing query is disposed before creating a new query
* All queries are disposed when the query execution service is disposed (ie, at shutdown of the service)
A unit test to verify the action of the dispose method for a ResultSet was added.
* Ensuring queries are disposed
Adding logic to dispose any queries when:
* URI that already has a query executes another query
* A request to dispose a query is submitted
* A request to cancel a query is submitted
* Small tweaks for cleanup of query execution service
* WIP for ability to localize cell values
* Changing how DateTimeOffsets are stored, getting unit tests going
* Reworking BufferFileStreamWriter to use dictionary approach
* Plumbing the DbCellValue type the rest of the way through
* Removing unused components to simplify contract
* Cleanup and making sure byte[] appears in parity with SSMS
* CR comments, small tweaks for optimizing LINQ
Fixing a bug from the unit tests on OSX/Unix where attempting to create a file with a name that's all whitespace succeeds when it should fail. This was passing in Windows because File.Open throws an ArgumentException when an all whitespace name is provided. In Unix systems, File.Open does not throw, causing the test to fail.
Solution is to check for whitespace in the sanity check.
* Strings sweep for connection service
* String sweep for credentials service
* String sweep for hosting
* String sweep for query execution service
* String sweep for Workspace service
* Renaming utility namespace to match standards
Renaming Microsoft.SqlTools.EditorServices.Utility to
Microsoft.SqlTools.ServiceLayer.Utility to match the naming changes done a
while back. Also renaming them on the files that use them
* Namespace change on reliable connection
* Adding the new resx and designer files
* Final bug fixes for srgen
Fixing flakey moq package name
* Removing todo as per @kevcunnane
* Adding using statements as per @llali's comment
* Fixing issues from broken unit tests
Note: This feature contains changes that will break the contract for
saving as CSV and JSON. On success, null is returned as a message instead
of "Success". Changes will be made to the vscode component to handle this
change.
* Adding SRGen to build scripts
Adding support for sr.strings files. sr.strings files can be converted
into .resx and designer cs files by running the SRGen Cake target
Adding a new target to Cake script to run SRGen. SRGen is pulled down as a
nuget package for Cake and executes from the .tools folder.
Adding Cake temp environment to the gitignore
Adding the cake scripts to the sqltoolsservice solution file
* Fixes from pull request comments
* Fix as per @llali
* Actual changes as per @llali
* removing platform type from netcore dependency
This was an issue where non-string types were being read back as strings (ie, int was read back as a string) resulting in the objects coming back being completely unintelligible values. It was a very simple issue where the column datatype wasn't being set correctly in our wrapper type.
* WIP for buffering in temporary file
* Adding support for writing to disk for buffering
* WIP - Adding file reader, factory for reader/writer
* Making long list use generics and implement IEnumerable
* Reading/Writing from file is working
* Removing unused 'skipValue' logic
* More tweaks to file buffer
Adding logic for cleaning up the temp files
Adding fix for empty/null column names
* Adding comments and cleanup
* Unit tests for FileStreamWrapper
* WIP adding more unit tests, and finishing up wiring up the output writers
* Finishing up initial unit tests
* Fixing bugs with long fields
* Squashed commit of the following:
commit df0ffc12a46cb286d801d08689964eac08ad71dd
Author: Benjamin Russell <beruss@microsoft.com>
Date: Wed Sep 7 14:45:39 2016 -0700
Removing last bit of async for file writing.
We're seeing a 8x improvement of file write speeds!
commit 08a4b9f32e825512ca24d5dc03ef5acbf7cc6d94
Author: Benjamin Russell <beruss@microsoft.com>
Date: Wed Sep 7 11:23:06 2016 -0700
Removing async wrappers
* Rolling back test code for Program.cs
* Changes as per code review
* Fixing broken unit tests
* More fixes for codereview
