Commit Graph

12 Commits

Author SHA1 Message Date
Benjamin Russell
795eba3da6 Edit Data Service (#241)
This is a very large change. I'll try to outline what's going on.

1. This adds the **EditDataService** which manages editing **Sessions**.
    1. Each session has a **ResultSet** (from the QueryExecutionService) which has the rows of the table and basic metadata about the columns 
    2. Each session also has an **IEditTableMetadata** implementation which is derived from SMO metadata which provides more in-depth and trustworthy data about the table than SqlClient alone can.
    3. Each session holds a list of **RowEditBase** abstract class implementations
        1. **RowUpdate** - Update cells in a row (generates `UPDATE` statement)
        2. **RowDelete** - Delete an entire row (generates `DELETE` statement)
        3. **RowCreate** - Add a new row (generates `INSERT INTO` statement)
    4. Row edits have a collection of **CellUpdates** that hold updates for individual cells (except for RowDelete)
        1. Cell updates are generated from text
     5. RowEditBase offers some baseline functionality
        1. Generation of `WHERE` clauses (which can be parameterized)
        2. Validation of whether a column can be updated
2. New API Actions
    1. edit/initialize - Queries for the contents of a table/view, builds SMO metadata, sets up a session
    2. edit/createRow - Adds a new RowCreate to the Session
    3. edit/deleteRow - Adds a new RowDelete to the Session
    4. edit/updateCell - Adds a CellUpdate to a RowCreate or RowUpdate in the Session
    5. edit/revertRow - Removes a RowCreate, RowDelete, or RowUpdate from the Session
    6. edit/script - Generates a script for the changes in the Session and stores to disk
    7. edit/dispose - Removes a Session and releases the query
3. Smaller updates (unit test mock improvements, tweaks to query execution service)

**There are more updates planned -- this is just to get eyeballs on the main body of code**

* Initial stubs for edit data service

* Stubbing out update management code

* Adding rudimentary dispose request

* More stubbing out of update row code

* Adding complete edit command contracts, stubbing out request handlers

* Adding basic implementation of get script

* More in progress work to implement base of row edits

* More in progress work to implement base of row edits

* Adding string => object conversion logic and various cleanup

* Adding a formatter for using values in scripts

* Splitting IMessageSender into IEventSender and IRequestSender

* Adding inter-service method for executing queries

* Adding inter-service method for disposing of a query

* Changing edit contract to include the object to edit

* Fully fleshing out edit session initialization

* Generation of delete scripts is working

* Adding scripter for update statements

* Adding scripting functionality for INSERT statements

* Insert, Update, and Delete all working with SMO metadata

* Polishing for SqlScriptFormatter

* Unit tests and reworked byte[] conversion

* Replacing the awful and inflexible Dictionary<string, string>[][] with a much better test data set class

* Fixing syntax error in generated UPDATE statements

* Adding unit tests for RowCreate

* Adding tests for the row edit base class

* Adding row delete tests

* Adding RowUpdate tests, validation for number of key columns

* Adding tests for the unit class

* Adding get script tests for the session

* Service integration tests, except initialization tests

* Service integration tests, except initialization tests

* Adding messages to sr.strings

* Adding messages to sr.strings

* Fixing broken unit tests

* Adding factory pattern for SMO metadata provider

* Copyright and other comments

* Addressing first round of comments

* Refactoring EditDataService to have a single method for handling
session-dependent operations
* Refactoring Edit Data contracts to inherit from a Session and Row
operation params base class
* Copyright additions
* Small tweak to strings
* Updated unit tests to test the refactors

* More revisions as per pull request comments
2017-02-22 17:32:57 -08:00
Benjamin Russell
08521cf61c Adding TestDbResultSet for Better Mocking (#231)
* Replacing the awful and inflexible Dictionary<string, string>[][] with a much better test data set class

* Applying all changes

* Removing unused RequestParamTests
2017-02-15 13:02:22 -08:00
Aditya Bist
eb4f2f2b91 port batch parser wrapper (#232)
* Initial commit for GitHub IO pages

* Add initial doxfx content

* Update manifest.json

* Update manifest.json

* Set theme jekyll-theme-cayman

* Set theme jekyll-theme-slate

* Set theme jekyll-theme-minimal

* Set theme jekyll-theme-tactile

* Clear out theme setting

* Remove API docs

* Revert "Adding Milliseconds to DateTime fields (#173)" (#197)

This reverts commit 431dfa4156.

* ported new BatchParser

* added BatchParser tests

* fixing merge conflicts

* fix build issues

* cleaned code and addressed comments from code review

* addressed code review and made BatchParser logic more efficient

* fixed batch parser tests

* changed class name to fix build issues

* fixed merge conflicts

* added path for lab mode baseline tests

* changed env path for lab mode

* added env variable to appveyor

* testing env variable for appveyor

* fixed lab build

* debug appveyor build

* testing changes for appveyor

* changed trace env path

* debugging appveyor build

* changed baseline env path

* debugging

* debugging

* debugging

* switched on trace flag

* debugging

* debugging

* changed build config

* changed baseline files

* checking baseline output

* changed baseline files

* debug baseline tests

* debugging baseline

* debugging

* debugging

* debug

* debugging

* testing baseline format

* debug

* debug

* debug

* debug

* debug

* newline debug

* changed baseline file

* debug

* test

* try new way to read

* added execution engine tests

* change test

* testing file encoding

* moved execution engine tests to integration

* try compare without spaces

* removed no op test

* added env variable for travis

* put batch parser tests to integration too

* put batch parser in integration

* try new baseline string match

* compare baseline test logic changed

* changed baseline logic as well as cleaned tests

* fix build for travis CI

* fix travis CI issues

* fixed highlighting bugs on vscode

* code review changes

* ported new BatchParser

* added BatchParser tests

* Initial commit for GitHub IO pages

* Add initial doxfx content

* Update manifest.json

* Update manifest.json

* Set theme jekyll-theme-cayman

* Set theme jekyll-theme-slate

* Set theme jekyll-theme-minimal

* Set theme jekyll-theme-tactile

* Clear out theme setting

* Remove API docs

* Revert "Adding Milliseconds to DateTime fields (#173)" (#197)

This reverts commit 431dfa4156.

* fixing merge conflicts

* fix build issues

* cleaned code and addressed comments from code review

* addressed code review and made BatchParser logic more efficient

* fixed batch parser tests

* changed class name to fix build issues

* fixed merge conflicts

* added path for lab mode baseline tests

changed env path for lab mode

added env variable to appveyor

testing env variable for appveyor

fixed lab build

debug appveyor build

testing changes for appveyor

changed trace env path

debugging appveyor build

changed baseline env path

debugging

debugging

debugging

switched on trace flag

debugging

debugging

changed build config

changed baseline files

checking baseline output

changed baseline files

debug baseline tests

debugging baseline

debugging

debugging

debug

debugging

testing baseline format

debug

debug

debug

debug

debug

newline debug

changed baseline file

debug

test

try new way to read

added execution engine tests

change test

testing file encoding

moved execution engine tests to integration

try compare without spaces

removed no op test

added env variable for travis

* put batch parser tests to integration too

* put batch parser in integration

try new baseline string match

* compare baseline test logic changed

* changed baseline logic as well as cleaned tests

* fix build for travis CI

* fix travis CI issues

* fixed highlighting bugs on vscode

* code review changes

* fixed filestream writer test

* added localization string

* added localization string

* generated new string files again

* code review changes
2017-02-10 16:51:26 -08:00
Benjamin Russell
54c43f950f Inter-Service API for executing queries (#223)
Adding new methods for executing queries from other services (such as the upcoming edit data service). The code is written to avoid duplicating logic by using lambdas to perform custom logic.
Additionally, the service host protocol has been slightly modified to split the IMessageSender into IEventSender and IRequestSender. This allows us to use either a ServiceHost or any RequestContext<T> to send events. It becomes very convenient to use another service's request context to send the events for query execution.

**Breaking Change**: This removes the messages property for query dispose results and instead elects to use error for any errors encountered during query disposal. A result is only used when something is successful.

* Splitting IMessageSender into IEventSender and IRequestSender

* Adding inter-service method for executing queries

* Adding inter-service method for disposing of a query

* Adding null checking for the success/error handlers
2017-02-02 17:05:10 -08:00
Benjamin Russell
857b526553 query/executeString (#217)
This is a small API addition that allows us to execute queries directly as strings. This will make it easier to execute queries outside the confines of a workspace like VS Code.

* Refactor execution requests and events are now named less redundantly and moved into a separate namespace for organization. This is the bulk of the changes.
    * QueryExecuteBatchNotification -> ExecuteRequests/BatchEvents
    * QueryExecuteMessageNotification -> ExecuteRequests/MessageEvent
    * QueryExecuteCompleteNotification -> ExecuteRequests/QueryCompleteEvent
    * QueryExecuteResultSetCompleteNotification -> ExecuteRequests/ResultSetEvents
    * QueryExecuteSubsetRequest -> SubsetRequest.cs
* Creating an inheritance pattern where
    * `ExecuteRequestParamsBase` has execution options and ID for a query execute request
    * `ExecuteDocumentSelectionParams` inherits from `ExecuteRequestParamsBase` and provides a document selection
    * `ExecuteStringParams`  inherits from `ExecuteRequestParamsBase` and provides the query text
* Adding a helper method to get SQL text based on request type
* Through the AWESOME POWER OF POLYMORPHISM, we are able to create a request for executing straight SQL basically for free.
* **Breaking change:** query/execute => query/executeDocumentSelection to make it more obvious what is expected.
* Adding unit tests for the code that gets SQL text

* Refactoring of execute contracts into their own namespace

* Refactoring application

* Adding new request for executing queries as strings

* Adding forgotten string request

* Changing the logic for checking the request param types

* Removing redundant declarations
2017-01-30 15:24:12 -08:00
Leila Lali
b353b2137e New test common project for database connections using the settings.json (#210)
* moved test driver tests and test common classes to separate projects
2017-01-11 13:47:56 -08:00
Benjamin Russell
e71bcefb28 Feature: Progressive Messages (#208)
This change is a reworking of the way that messages are sent to clients from the service layer. It is also a reworking of the protocol to ensure that all formulations of query send back events to the client in a deterministic ordering. To support the first change:
* Added a new event that will be sent when a message is generated
* Messages now indicate which Batch (if any) generated them
* Messages now indicate if they were error level
* Removed message storage in Batch objects and BatchSummary objects
* Batch objects no longer have error state
2017-01-10 16:42:03 -08:00
Benjamin Russell
1e59166147 Event Flow Validator (#199)
This is a reworking of the unit tests to permit us to better test events from the service host. This new Event Flow Validator class allows creating a chain of events that can then be validated after execution of the request. Each event can have its own custom validation logic for verifying that the object sent via the service host is correct. It also allows us to validate that the order of events are correct.

The big drawback is that (at this time) the validator cannot support asynchronous events or non-determinant ordering of events. We don't need this for the query execution functionality despite messages being sent asynchronously because async messages aren't sent during unit tests (due to the db message event only being present on SqlDbConnection classes). If the need arises to do async or out of order event validation, then I have some ideas for how we can do that.

* Applying the event flow validator to the query execution service integration tests

* Undoing changes to events that were included in cherry-picked commit

* Cleaning up event flow validation to query execution

* Add efv to cancel tests

* Adding efv to dispose tests

* Adding efv to subset tests

* Adding efv to SaveResults tests

* Copyright
2016-12-20 11:59:03 -08:00
Benjamin Russell
54f30887cc Batch Start Notification (#169)
This change is part of the progressive results code. It will submit a notification from the service layer to indicate when execution of a batch has completed. This notification will contain the selection for batch, execution start time, and its ID. This will enable the extension to produce a header for the batch before the batch completes, in order to make it more clear to the user that execution is going on.

* Adding new event for batch start

* Unit tests

* Fixing comments as per @kevcunnane
2016-12-08 11:23:08 -08:00
Benjamin Russell
7de742bfab Fixing race condition in unit test (#161) 2016-11-23 13:34:48 -08:00
Benjamin Russell
d9efb95386 Progressive Results Part 2: Result Completion Event (#134)
The main change in this pull request is to add a new event that will be fired upon completion of a resultset but before the completion of a batch. This event will only fire if a resultset is available and generated.

Changes:
* ConnectionService - Slight changes to enable mocking, cleanup 
* Batch - Moving summary generation into ResultSet class, adding generation of ordinals for resultset and locking of result set list (which needs further refinement, but would be outside scope of this change)
* Adding new event and associated parameters for completion of a resultset. Params return the resultset summary
* Adding logic for assigning the event a handler in the query execution service
* Adding unit tests for testing the new event /making sure the existing tests work
* Refactoring some private properties into member variables

* Refactor to remove SectionData class in favor of BufferRange

* Adding callback for batch completion that will let the extension know that a batch has completed execution

* Refactoring to make progressive results work as per async query execution

* Allowing retrieval of batch results while query is in progress

* reverting global.json, whoops

* Adding a few missing comments, and fixing a couple code style bugs

* Using SelectionData everywhere again

* One more missing comment

* Adding new notification type for result set completion

* Plumbing event for result set completion

* Unit tests for result set events

This includes a fairly substantial change to create a mock of the
ConnectionService and to create separate memorystream storage arrays. It
preserves more correct behavior with a integration test, fixes an issue
where the test db reader will return n-1 rows because the Reliable
Connection Helper steals a record.

* Adding locking to ResultSets for thread safety

* Adding/fixing unit tests

* Adding batch ID to result set summary
2016-11-22 17:37:27 -08:00
Benjamin Russell
ec94d986a8 Unit Test Cleanup (#141)
This is a fairly large set of changes to the unit tests that help isolate the effectiveness of the unit tests.

* Unit tests for query execution have been split into separate files for different classes.
* Unit tests have been added for the ResultSet class which previously did not have tests
* The InMemoryStreamWrapper has been improved to share memory, creating a simulated filesystem
* Creating a mock ConnectionService to decrease noisy exceptions and prevent "row stealing". Unfortunately this lowers code coverage. However, since the tests that touched the connection service were not really testing it, this helps keep us honest. But it will require adding more unit tests for connection service.
* Standardizing the await mechanism for query execution
* Cleaning up the mechanism for getting WorkspaceService mocks and mock FileStreamFactories

* Refactor the query execution tests into their own files

* Removing tests from ExecuteTests.cs that were moved to separate files

* Adding tests for ResultSet class

* Adding test for the FOR XML/JSON component of the resultset class

* Setting up shared storage between file stream readers/writers

* Standardizing on Workspace mocking, awaiting execution completion

* Adding comment for ResultSet class
2016-11-10 11:42:31 -08:00