Commit Graph

64 Commits

Author SHA1 Message Date
Aasim Khan
2e7bac5659 Sending show plan graph to ADS on Result Set updated event (#1300)
* Sending showplan graph over json rpc in Result updated event
Translating showplan graph into simple objects to be sent over JSON RPC

* Revert "Sending showplan graph over json rpc in Result updated event"

This reverts commit 2d63a625fd200d057bf6093e233f05dea440347c.

* Added string for localization

* Sending showplan graph over json rpc in Result updated event
Translating showplan graph into simple objects to be sent over JSON RPC

* Refactoring class

* Removing test warning

* Removing unused imports
Adding copyright

* Removing unused prop

* removing formatted string out .strings file

* Formatting files
Adding Errors in show plan graph

* Adding a separate event for execution plan

* Now sending mulitple graphs when a batch has more than one query.
2021-11-16 22:33:28 -08:00
Alex Ma
431e4cfbf1 Add changeConnectionUri action for query (#1227)
* added wip rename query

* added first rename test

* added missing query handling and two new tests

* removed ServiceDispose

* added change to owneruri.

* made fields private, used accessor methods

* added readonly flag back

* added didSave to sqltools.servicelayer

* fix for docsavecallbacks

* removed tryremove

* test disconnect in queryExecutionService

* removed debug message working query.

* added comment to handlerenamerequest

* added uri replacement function

* removed duplicate removemap function

* added minor fixes

* changed connectionOwnerUri to getter setter

* removed handledidsave and added setters

* removed additional save parts

* fixed space issues

* more fixes for spaces

* restored spaces

* fixed workspaceservice

* Update TextDocument.cs

* changed renameRequest into notification WIP

* restored textdocument

* restored textdocument.cs

* added rightwards arrow

* renamed rename to ChangeConnectionUri

* added more renames

* rename changeconnectionuri to changeuri

* renamed file names

* Revert "renamed file names"

This reverts commit 55228e65025b5179b15ae9a0adc095d95538723d.

* Revert "rename changeconnectionuri to changeuri"

This reverts commit 23f3813f609e3947f103e057f7c0919184075bdc.

* removed logging message and other small changes

* renamed class and method
2021-08-30 11:06:07 -07:00
Charles Gagnon
dc6555a823 Escape file URIs (#1189)
* Escape file URIs

* Fix test

* Add tests
2021-04-16 15:33:11 -07:00
Amir Omidi
e95b4c84c5 Add execution options to the stream factory (#848) 2019-10-03 10:45:23 -07:00
Karl Burtram
151a2de625 Query execution settings support (#812)
* WIP

* WIP 3

* WIP 3

* Additional query settings updates

* Add settings tets

* Address code review feeback

* Updates from testing
2019-05-16 15:21:17 -07:00
udeeshagautam
c86f43618c Proposed fix for : 3653 : Cancel not working even after disconnecting (#797)
* Proposed changes to stop query execution if an exception has occured before/during batch execution

* Sending the error message was causing a test to fail - so removing for now. We can update the test to expect this is needed.
2019-05-14 16:15:09 -07:00
udeeshagautam
5778dc7b01 Fix for : 4045: Cannot cancel a Query. Query runs too long. (And SMO update) (#780)
* Fix for : 4045: Cannot cancel a Query. Query runs too long.
HandleExecuteRequest was returning a task to awaiter - which was getting waited on. changed it to async function to start task in new thread and return nothing to awaiter . Also the cancellation token set by cancel request was getting checked only after making the connection to execute batches. Added an additional check for cancellation token befor the connection has been made.

Fix for 4319: Error showing dbs when using AAD in... 1.5.0-alpha.74
David has already created a new version of SMO nuget with the fix. - incorporating the same (150.18096.0-preview).

* Adding awaitable internal task for tests to run properly

* Adding more cancel tests
2019-03-13 15:39:00 -07:00
Arvind Ranasaria
6dd9a4b5f1 Feat/result streaming (#721)
This changes adds the following two notifications from the results processing within a batch. These new notifications allows a consumer to stream results from a resultset instead of getting them all at once after the entire resultset has been fetched.

ResultsAvailable
This is issued after at least 1 row has been fetched for this resultset.

ResultsUpdated
This is issued periodically as more rows are available on this resultset. The final send of this notification when all rows have been fetched has the property 'Complete' set to true in the ResultSummary object.

Detailed Change Log:
* Initial completed implementation of QueryResults stream feature. 3 unittests still need fixing

* Fix for the 3 failing test. I will look into making MockBehavior strict again for the three tests later

* Making GetReader/GetWriter use filestream objects in FileShare.ReadWrite mode so the file can be concurrently read and written

* Changing resultsAvailable also to fire off on a timer instead of after 1st row

* adding a project for clr TableValuedFunction to produce result set with delays after each row. This is helpful in end to end testing.

* Fixing up some tests and simplifying implementation of result update timer

* Address review comments

* Some test fixes

* Disabled flaky test verification
2018-11-26 10:24:54 -08:00
ranasaria
09652cccd1 Enhanced Logging for sqltoolsservice (#695)
This change modifies the logging framework within sqltoolservice.
Moves away from custom Logger object to start using .Net tracing framework. It supports for the static Trace and TraceSource way of logging. For all new code it is recommend that we log the log messages using the existing static Logger class, while the code changes will continue to route the older Trace.Write* calls from the process to same log listeners (and thus the log targets) as used by the Logger class. Thus tracing in SMO code that uses Trace.Write* methods gets routed to the same file as the messages from rest of SQLTools Service code.
Make changes to start using .Net Frameworks codebase for all logging to unify our logging story.
Allows parameter to set tracingLevel filters that controls what kinds of message make it to the log file.
Allows a parameter to set a specific log file name so if these gets set by external code (the UI code using the tools service for example) then the external code is aware of the current log file in use.
Adding unittests to test out the existing and improved logging capabilities.


Sequences of checkins in development branch:
* Saving v1 of logging to prepare for code review. Minor cleanup and some end to end testing still remains
* Removing local launchSettings.json files
* added support for lazy listener to sqltoolsloglistener and removed incorrect changes to comments across files in previous checkin
* Converting time to local time when writing entries to the log
* move the hosting.v2 to new .net based logging code
* removing *.dgml files and addding them to .gitignore
* fixing typo of defaultTraceSource
* Addressing pull request feedback
* Adding a test to verify logging from SMO codebase
* propogating changes to v1 sqltools.hosting commandoptions.cs to the v2 version
* Fixing comments on start and stop callstack methods and whitespaces
* Commenting a test that got uncommented by mistake
* addding .gitattributes file as .sql file was observed to be misconstrued as a binary file
2018-09-24 23:55:59 -07:00
Mustafa Sadedil
98018c5292 Save as XML feature added (#684)
Similar approach used like Save as JSON or Save as CSV.
2018-09-10 12:11:51 -07:00
Aditya Bist
7c7395fce0 Added support for T-SQL parse (#642)
* added support for t-sql parse

* added syntax parse to language service instead

* fixed misleading error

* code review comments
2018-06-26 14:19:05 -07:00
Cory Rivera
f57108f198 Add flag so that full column schema is only fetched with EditData queries. (#615) 2018-05-03 13:38:08 -07:00
Kevin Cunnane
185978eb80 Fix empty result set error handling and metadata service returning correct type name (#535)
- HandleSimpleExecuteRequest now handles the case where no rows are in a result by cleanly returning a success message but with no rows included. This is handled in the front-end instead and goes through the standard path (with a clean explanation message) instead of showing a `error: no results to return`
- MetadataService was always meant to include the type name in the return result, as otherwise the front end has to guess. In order to fix a bug where this resulted in scripting based on the metadata failing (as front-end used `Procedure` instead of `StoredProcedure`), I'm returning the data here. I'll have a matching front end fix but this is overall a good solution to have.
2017-10-30 11:36:46 -07:00
Kevin Cunnane
3392f93a2e Add rowcount check so that no results error is sent for a query where no rows are returned (#525) 2017-10-25 12:06:22 -07:00
Henry Phan
784f4c5d05 Feature/serialization exe (#442)
* Initial changes to have serialization generate its own .exe

* Removed additional project from sln file

* remove all references to removed temporary project

* Moved shared contracts into own dll and fixed imports. Addressed PR comments

* Undid having a separate contracts project since that'll be a task for later on. Moved dbcellvalue and saveresultsrequest to Hosting, where they will be imported and shared by the service layer and serialization projects

* Switched backslashes in project reference in csproj file to forward slashes for consistency

* Moved necessary contracts back to service layer. Refactored CommandOptions to reduce code duplication. Addressed miscellaneous PR suggestions

* Accidentally left these files out of previous commit

* Initialized loggers for serialization and credentials with the logging directory provided by the cmd line arg, if there is one

* Changed default log directory paths for serialization and credentials. Removed unnecessary cast and added a copyright

* Changed name of generated executable for serialization service

* removed unnecessary object cast

* removing unnecessary imports and addressing other PR comments
2017-09-05 16:21:42 -07:00
Anthony Dresser
7ef81d0e54 Multiple Connection Simple Execute (#421)
* change simple execute to open a new connection and close it every query

* updated tests for simple execute

* removed an unnecessary connect

* refactored code to be more readable

* global try catch on simple execute

* added multiple execution test

* update execution to be asynchrous; update tests to account for asynchrounous nature
2017-07-28 13:35:46 -07:00
Karl Burtram
cfa78bbc3b Execute SQL statement at cursor location (#412)
* Stage changes to other machine

* Parse sql statement from script document

* Fix a few typos and minor changes

* Fix bug
2017-07-13 20:33:05 -07:00
Kevin Cunnane
3aba287759 Fix #911 handle perforce files (#400)
- Additional handling of document events with "perforce:" or other SCM strings
- Much of the handling had been added already, but adding in additional validation and ensuring that everywhere `Workspace.GetFile` is called we add a not-null check to avoid null reference errors
2017-07-05 13:25:05 -07:00
Anthony Dresser
af2ed84953 Adds a execute and return result message (#383)
* inital request

* refactored query execution failure callback to take exception

* added failure callback to execute and return

* added test for query execute and return

* updated params

* removed dead code

* addressed feedback; added multiple active result set support; updated tests

* addessed feedback and added testing and errors and verification

* change <= to ==

* changed name of trashQ to removedQuery
2017-06-16 15:43:41 -07:00
Anthony Dresser
9b91a93578 refactored query execution failure callback to take exception (#382) 2017-06-14 15:59:21 -07:00
Benjamin Russell
2eb60f45c9 Send Error Object on SendError (#304)
This change ensures that when calling `requestContext.SendError` you are only able to supply parameters that match the language service beta protocol expected Error object. In other words, you have to provide an error message and optionally and error code.

# **BREAKING API CHANGES**
This will break displaying errors in Microsoft/vscode-mssql. I will be making changes to properly handle the error object shortly.

* Adding contract for returning Error objects as per LanguageService "protocol"

* Fixes throughout codebase to send only error message in error cases
Cleanup of CredentialServiceTest unit test class
Adding standard error handling for event flow validator

* Adding optional data field as per protocol spec
https://github.com/Microsoft/language-server-protocol/blob/master/protocol.md

* Adding optional validation for error objects
2017-04-05 14:47:37 -07:00
Benjamin Russell
d7ecfb1a87 feature/edit/subset (#283)
* Changing query/subset API to only use Result on success, Error on error

* Creating an interservice API for getting query result subsets

* Updates to subset API

* RowStartIndex is now long
* Output of query/subset is a 2D array of DbCellValue
* Adding LongSkip method to LongList to allow skipping ahead by longs
* Moving LongList back to ServiceLayer utilities. Move refactoring

* Stubbing out request for edit/subset

* Initial implementation of getting edit rows

* Unit tests for RowEdit and RowDelete .GetEditRow

* Fixing major bugs in LongList implementation, adding much more thorough tests

* Adding some more unit tests and fixes to make unit tests pass

* Fixing comment
2017-03-21 15:14:04 -07:00
Wujun Zhou
8d47d5c7b3 Save As Excel (#279)
* Fix dispose pattern usage in SaveAsWriterBase

* Add SaveAsExcel feature

This adds the save as excel function to the backend. To reduce large dependency and run on dotnet core now, this implementation use a raw excel writer (the SaveAsExcelFileStreamWriterHelper.cs) instrad of popular excel library, such as EPPlus or OpenXmlSdk.

* Fix can not open the generated excel file in google sheet

For the file name inside excel, google uses a case sensitive path while
Excel doesn't. This change fix the case, so that the file name matches
the one in x1/_rels/workbook.xml.rels

* Fix datetime doesn't recognized by google sheet

Google doesn't support cell type t="d" with ISO 8601 date. (From stackoverflow thread and testing), thus use the old way of excel datetime, which uses double to present datetime

* update to use xmlwriter

* Add basic unit tests for SaveAsExcelFileStreamWriterHelper

* refactor: simplify the public interface of the SaveAsExcelFileStreamWriterHelper

* update private fields names based on the name convention

* Add comments to classes of SaveAsExcel feature

* clean up SaveAsExcelFileStreamWriterHelper

- change SaveAsExcelFileStreamWriterHelper from public to internal
- remove the PenddingRowEndTag function from referenceManager
- change the SaveAsExcelFileStreamWriterHelper(stream) to default leaveOpen to false to match the normal behavior
- change the rowreference to use XmlConvert to convert int to string
- rename writeSetting to writerSetting and add private

* fix CI test error for SaveAsExcel

* remove ExporterException in SaveAsExcel

* fix lefe over CSV to Excel in the comments

* refactor to be consistent with JsonWriter and remove the comment

* remove commented out test

The test is too slow to run

* fix typo in comment

* refactor SaveAsExcel to the coding standard

* refactor rewrite the WriteStyle with XmlWriter

* Add licence header

* reverse mistakenly checked-in changes

* fix: left-over CSV in commets

* remove duplicate check

The check was done in the IncreaseColumnReference, but that check is too late in case of too many columns. All the addCell do the check at the begining now

* fix TimeSpan more than 24 hours

* fix AddRowMustBeCalledBeforeAddCellException test

This is  due to remove duplicate call to AssureColumnReference in  WriteAndIncreaseColumnReference

* fix: TimeSpan will write twice

* style: change retun in the switch to break

* Add bool format

* remove todo in comment

This provides extra safeguard in the cost of one memory access when null.
2017-03-16 14:33:12 -07:00
Karl Burtram
f00136cffb Fix namespaces for code moved to separate class libraries (#263)
* Create ServiceHost a service-specific class

* Renaming hosting namespace

* Rename credentials namespace

* Fix namespaces
2017-03-02 14:41:06 -08:00
Benjamin Russell
795eba3da6 Edit Data Service (#241)
This is a very large change. I'll try to outline what's going on.

1. This adds the **EditDataService** which manages editing **Sessions**.
    1. Each session has a **ResultSet** (from the QueryExecutionService) which has the rows of the table and basic metadata about the columns 
    2. Each session also has an **IEditTableMetadata** implementation which is derived from SMO metadata which provides more in-depth and trustworthy data about the table than SqlClient alone can.
    3. Each session holds a list of **RowEditBase** abstract class implementations
        1. **RowUpdate** - Update cells in a row (generates `UPDATE` statement)
        2. **RowDelete** - Delete an entire row (generates `DELETE` statement)
        3. **RowCreate** - Add a new row (generates `INSERT INTO` statement)
    4. Row edits have a collection of **CellUpdates** that hold updates for individual cells (except for RowDelete)
        1. Cell updates are generated from text
     5. RowEditBase offers some baseline functionality
        1. Generation of `WHERE` clauses (which can be parameterized)
        2. Validation of whether a column can be updated
2. New API Actions
    1. edit/initialize - Queries for the contents of a table/view, builds SMO metadata, sets up a session
    2. edit/createRow - Adds a new RowCreate to the Session
    3. edit/deleteRow - Adds a new RowDelete to the Session
    4. edit/updateCell - Adds a CellUpdate to a RowCreate or RowUpdate in the Session
    5. edit/revertRow - Removes a RowCreate, RowDelete, or RowUpdate from the Session
    6. edit/script - Generates a script for the changes in the Session and stores to disk
    7. edit/dispose - Removes a Session and releases the query
3. Smaller updates (unit test mock improvements, tweaks to query execution service)

**There are more updates planned -- this is just to get eyeballs on the main body of code**

* Initial stubs for edit data service

* Stubbing out update management code

* Adding rudimentary dispose request

* More stubbing out of update row code

* Adding complete edit command contracts, stubbing out request handlers

* Adding basic implementation of get script

* More in progress work to implement base of row edits

* More in progress work to implement base of row edits

* Adding string => object conversion logic and various cleanup

* Adding a formatter for using values in scripts

* Splitting IMessageSender into IEventSender and IRequestSender

* Adding inter-service method for executing queries

* Adding inter-service method for disposing of a query

* Changing edit contract to include the object to edit

* Fully fleshing out edit session initialization

* Generation of delete scripts is working

* Adding scripter for update statements

* Adding scripting functionality for INSERT statements

* Insert, Update, and Delete all working with SMO metadata

* Polishing for SqlScriptFormatter

* Unit tests and reworked byte[] conversion

* Replacing the awful and inflexible Dictionary<string, string>[][] with a much better test data set class

* Fixing syntax error in generated UPDATE statements

* Adding unit tests for RowCreate

* Adding tests for the row edit base class

* Adding row delete tests

* Adding RowUpdate tests, validation for number of key columns

* Adding tests for the unit class

* Adding get script tests for the session

* Service integration tests, except initialization tests

* Service integration tests, except initialization tests

* Adding messages to sr.strings

* Adding messages to sr.strings

* Fixing broken unit tests

* Adding factory pattern for SMO metadata provider

* Copyright and other comments

* Addressing first round of comments

* Refactoring EditDataService to have a single method for handling
session-dependent operations
* Refactoring Edit Data contracts to inherit from a Session and Row
operation params base class
* Copyright additions
* Small tweak to strings
* Updated unit tests to test the refactors

* More revisions as per pull request comments
2017-02-22 17:32:57 -08:00
Benjamin Russell
2b15890b00 Exposing the DisplayBitAsNumber setting (#248)
* Exposing the DisplayBitAsNumber setting

* Properly wiring up the settings for query execution service

* Fixing broken unit test
2017-02-22 14:31:29 -08:00
Benjamin Russell
55a56be316 Adding new setting for numeric or text bit display (#243)
Adding a new setting to query execution setting that will change the display value we generate for `BIT` columns. The new setting is `DefaultDisplayBitAsNumber`. If true, bit columns will be displayed as 1 or 0. If false, they'll be displayed as true or false. The default value is true, to keep parity with SSMS behavior.

Enables us to solve https://github.com/Microsoft/vscode-mssql/issues/690 and https://github.com/Microsoft/vscode-mssql/issues/513
2017-02-21 19:29:55 -08:00
Benjamin Russell
54c43f950f Inter-Service API for executing queries (#223)
Adding new methods for executing queries from other services (such as the upcoming edit data service). The code is written to avoid duplicating logic by using lambdas to perform custom logic.
Additionally, the service host protocol has been slightly modified to split the IMessageSender into IEventSender and IRequestSender. This allows us to use either a ServiceHost or any RequestContext<T> to send events. It becomes very convenient to use another service's request context to send the events for query execution.

**Breaking Change**: This removes the messages property for query dispose results and instead elects to use error for any errors encountered during query disposal. A result is only used when something is successful.

* Splitting IMessageSender into IEventSender and IRequestSender

* Adding inter-service method for executing queries

* Adding inter-service method for disposing of a query

* Adding null checking for the success/error handlers
2017-02-02 17:05:10 -08:00
Benjamin Russell
857b526553 query/executeString (#217)
This is a small API addition that allows us to execute queries directly as strings. This will make it easier to execute queries outside the confines of a workspace like VS Code.

* Refactor execution requests and events are now named less redundantly and moved into a separate namespace for organization. This is the bulk of the changes.
    * QueryExecuteBatchNotification -> ExecuteRequests/BatchEvents
    * QueryExecuteMessageNotification -> ExecuteRequests/MessageEvent
    * QueryExecuteCompleteNotification -> ExecuteRequests/QueryCompleteEvent
    * QueryExecuteResultSetCompleteNotification -> ExecuteRequests/ResultSetEvents
    * QueryExecuteSubsetRequest -> SubsetRequest.cs
* Creating an inheritance pattern where
    * `ExecuteRequestParamsBase` has execution options and ID for a query execute request
    * `ExecuteDocumentSelectionParams` inherits from `ExecuteRequestParamsBase` and provides a document selection
    * `ExecuteStringParams`  inherits from `ExecuteRequestParamsBase` and provides the query text
* Adding a helper method to get SQL text based on request type
* Through the AWESOME POWER OF POLYMORPHISM, we are able to create a request for executing straight SQL basically for free.
* **Breaking change:** query/execute => query/executeDocumentSelection to make it more obvious what is expected.
* Adding unit tests for the code that gets SQL text

* Refactoring of execute contracts into their own namespace

* Refactoring application

* Adding new request for executing queries as strings

* Adding forgotten string request

* Changing the logic for checking the request param types

* Removing redundant declarations
2017-01-30 15:24:12 -08:00
Raymond Martin
8a8d4338f1 Feature execution plan settings and request implementation (#213)
* experimental showplan implementation (tools side only)

* fix for redundant massages from showplan executions

* moved showplan batches to new variables to make it less confusing

* returns showplan as part of batch summary with in each result summary

* cleaned up showplan resultsets

* cleaning up code and making showplan var optional

* changes all var names to showplan

* adding estimated support

* small fixes

* updatin var names and adding EPOptions struct

* adding ssms execution plan logic based on server types

* adding special actions logic

* removing redundant name changes

* execution plan query handler added

* cleaning up functions and data structures

* seperated special actions into its own class

* cleaning up special actions

* cleaning up code

* small new line fixes

* commenting out pre-yukon code

* removing all pre yukon code

* last yukon code commented out

* fixes broken tests

* adding related unit tests; integration tests incoming

* finishing tests and cleaning up code

* semantic changes

* cleaning up semantics

* changes and test fixes, also adding new exceptions into RS

* fixing special actions and cleaning up request logic

* fixing comment to trigger new build

* triggering another  build

* fixed up specialaction and added tests for it
2017-01-17 19:37:42 -08:00
Benjamin Russell
e71bcefb28 Feature: Progressive Messages (#208)
This change is a reworking of the way that messages are sent to clients from the service layer. It is also a reworking of the protocol to ensure that all formulations of query send back events to the client in a deterministic ordering. To support the first change:
* Added a new event that will be sent when a message is generated
* Messages now indicate which Batch (if any) generated them
* Messages now indicate if they were error level
* Removed message storage in Batch objects and BatchSummary objects
* Batch objects no longer have error state
2017-01-10 16:42:03 -08:00
Benjamin Russell
a8d627b07c Fixing by moving the result submission to before execution starts (#204) 2016-12-30 13:30:05 -08:00
Benjamin Russell
7ea1b1bb87 Move Save As to ResultSet (#181)
It's an overhaul of the Save As mechanism to utilize the file reader/writer classes to better align with the patterns laid out by the rest of the query execution. Why make this change? This change makes our code base more uniform and adherent to the patterns/paradigms we've set up. This change also helps with the encapsulation of the classes to "separate the concerns" of each component of the save as function. 

* Replumbing the save as execution to pass the call down the query stack as QueryExecutionService->Query->Batch->ResultSet
    * Each layer performs it's own parameter checking
        * QueryExecutionService checks if the query exists
        * Query checks if the batch exists
        * Batch checks if the result set exists
        * ResultSet checks if the row counts are valid and if the result set has been executed
    * Success/Failure delegates are passed down the chain as well
* Determination of whether a save request is a "selection" moved to the SaveResultsRequest class to eliminate duplication of code and creation of utility classes
* Making the IFileStream* classes more generic
    * Removing the requirements of max characters to store from the GetWriter method, and moving it into the constructor for the temporary buffer writer - the values have been moved to the settings and given defaults
    * Removing the individual type writers from IFileStreamWriter
    * Removing the individual type writers from IFIleStreamReader
* Adding a new overload for WriteRow to IFileStreamWriter that will write out data, given a row's worth of data and the list of columns
* Creating a new IFileStreamFactory that creates a reader/writer pair for reading from the temporary files and writing to CSV files
* Creating a new IFileStreamFactory that creates a reader/writer pair for reading from the temporary files and writing to JSON files
* Dramatically simplified the CSV encoding functionality
* Removed duplicated logic for saving in different types and condensed down to a single chain that only differs based on what type of factory is provided
* Removing the logic for managing the list of save as tasks, since the ResultSet now performs the actual saving work, there's no real need to expose the internals of the ResultSet
* Adding new strings to the sr.strings file for save as error messages
* Completely rewriting the unit tests for the save as mechanism. Very fine grained unit tests now that should cover majority of cases (aside from race conditions)


* Refactoring maxchars params into settings and out of file stream factory

* Removing write*/read* methods from file stream readers/writers

* Migrating the CSV save as to the resultset

* Tweaks to unit testing to eliminate writing files to disk

* WIP, moving to a base class for save results writers

* Everything is wired up and compiles

* Adding unit tests for CSV encoding

* Adding unit tests for CSV and Json writers

* Adding tests to the result set for saving

* Refactor to throw exceptions on errors instead of calling failure handler

* Unit tests for batch/query argument in range

* Unit tests

* Adding service integration unit tests

* Final polish, copyright notices, etc

* Adding NULL logic

* Fixing issue of unicode to utf8

* Fixing issues as per @kburtram code review comments

* Adding files that got broken?
2016-12-21 17:52:34 -08:00
Benjamin Russell
54f30887cc Batch Start Notification (#169)
This change is part of the progressive results code. It will submit a notification from the service layer to indicate when execution of a batch has completed. This notification will contain the selection for batch, execution start time, and its ID. This will enable the extension to produce a header for the batch before the batch completes, in order to make it more clear to the user that execution is going on.

* Adding new event for batch start

* Unit tests

* Fixing comments as per @kevcunnane
2016-12-08 11:23:08 -08:00
Mitchell Sternke
d1b791805a Fixed issue where no-op queries would never complete (#157)
* Fixed issue where no-op queries would never complete

* Changed approach to solving the race condition with no-op queries

* Addressing feedback
2016-11-28 14:28:05 -08:00
Karl Burtram
39b79ae05b Merge branch 'master' into dev 2016-11-28 09:58:32 -08:00
Benjamin Russell
d9efb95386 Progressive Results Part 2: Result Completion Event (#134)
The main change in this pull request is to add a new event that will be fired upon completion of a resultset but before the completion of a batch. This event will only fire if a resultset is available and generated.

Changes:
* ConnectionService - Slight changes to enable mocking, cleanup 
* Batch - Moving summary generation into ResultSet class, adding generation of ordinals for resultset and locking of result set list (which needs further refinement, but would be outside scope of this change)
* Adding new event and associated parameters for completion of a resultset. Params return the resultset summary
* Adding logic for assigning the event a handler in the query execution service
* Adding unit tests for testing the new event /making sure the existing tests work
* Refactoring some private properties into member variables

* Refactor to remove SectionData class in favor of BufferRange

* Adding callback for batch completion that will let the extension know that a batch has completed execution

* Refactoring to make progressive results work as per async query execution

* Allowing retrieval of batch results while query is in progress

* reverting global.json, whoops

* Adding a few missing comments, and fixing a couple code style bugs

* Using SelectionData everywhere again

* One more missing comment

* Adding new notification type for result set completion

* Plumbing event for result set completion

* Unit tests for result set events

This includes a fairly substantial change to create a mock of the
ConnectionService and to create separate memorystream storage arrays. It
preserves more correct behavior with a integration test, fixes an issue
where the test db reader will return n-1 rows because the Reliable
Connection Helper steals a record.

* Adding locking to ResultSets for thread safety

* Adding/fixing unit tests

* Adding batch ID to result set summary
2016-11-22 17:37:27 -08:00
Benjamin Russell
d5fbebc287 Progressive Results Part 1: Batch Completion Notification (#95)
The main feature of this pull request is a new callback that's added to the query class that is called when a batch has completed execution and retrieval of results. This callback will send an event to the extension with the batch summary information. After that, the extension can submit subset requests for the resultsets of the batch.
Other smaller changes in this pull request:
Refactor to assign a batch a id when its created instead of when returning the list of batch summaries
Passing the SelectionData around instead of extracting the values for it
Moving creation of BatchSummary into the Batch class
Retrieval of results is now permitted even if the entire query has not completed, as long as the batch requested has completed.
Also note, this does not break the protocol. It adds a new event that a queryRunner can listen to, but it doesn't require it to be listened to.

* Refactor to remove SectionData class in favor of BufferRange

* Adding callback for batch completion that will let the extension know that a batch has completed execution

* Refactoring to make progressive results work as per async query execution

* Allowing retrieval of batch results while query is in progress

* reverting global.json, whoops

* Adding a few missing comments, and fixing a couple code style bugs

* Using SelectionData everywhere again

* One more missing comment
2016-11-02 17:43:38 -07:00
Sharon Ravindran
6937e46c1b Handle connection exceptions and add error callback (#126) 2016-11-01 12:28:14 -07:00
Sharon Ravindran
29b7854ec1 Handle connection exceptions and add error callback (#126) 2016-10-31 11:25:51 -07:00
Sharon Ravindran
a27c182a0f Refactor error messages (#122) 2016-10-29 20:08:04 -07:00
Sharon Ravindran
931235c604 Refactor error messages (#122) 2016-10-28 12:26:31 -07:00
Sharon Ravindran
2a688cb87f Make save result async (#107)
* Make save results asynchronous

* Prevent write share of file

* Lock objects in stages

* Create Save result objects

* refactor and write rows in batches

* CHange batchSize from test value

* Remove await in handler

* Removing the file reader as a member of the resultset

* Change Dispose to wait for save

* Change concurrentBag

* PascalCase variables

* Modify function signature and tests

* Safe file methods

* refactor ResultSets to Ilist and remove ToList

* Change dictionary key and prevent add to saveTasks during dispose

* Simplify row concatenation

* Fix prevent add

* Fix prevent add

* Add methods to expose saveTasks and isBeingDisposed
2016-10-21 20:07:21 -07:00
Benjamin Russell
fa7e13d9c5 Cancel Query Part 2 (#89)
Moving some logic around such that when a query is cancelled, it isn't thrown away, allowing reading of the partial results, a la SSMS.
Adding a configure await to fix a tenacious bug causing query cancellations to hang for 20s or more.
Capturing sql errors for user cancellation, to return all user cancelation scenarios using the same messages. (ie, cancelling during ExecuteReaderAsync will yield a error from the server whereas cancelling during ReadAsync throws a TaskCancelledException
No changes to protocol, just implementation changes.

* Test of try/finally

* Fixed issue where resultsets are unreadable after query cancellation

* Fix for await/async issue
2016-10-12 17:41:48 -07:00
Benjamin Russell
60edcc3057 Make query execution truly asynchronous (#83)
The two main changes in this pull request:
Launching query execution as an asynchronous task that performs a callback upon completion or failure of a query. (Which also sets us up for callbacks progressive results)
Moving away from using the Result of a query execution to return an error. Instead we'll use an error event to return an error
Additionally, some nice refactoring and cleaning up of the unit tests to take advantage of the cool RequestContext mock tooling by @kevcunnane

* Initial commit of refactor to run execution truely asynchronously

* Moving the storage of the task into Query class

Callbacks for completion of a query and failure of a query are setup as
events in the Query class. This actually sets us up for a very nice
framework for adding batch and resultset completion callbacks.

However, this also exposes a problem with cancelling queries and returning
errors -- we don't properly handle errors during execution of a query
(aside from DB errors).

* Wrapping things up in order to submit for code review

* Adding fixes as per comments
2016-10-11 10:51:52 -07:00
Benjamin Russell
1b8e9c1e86 Lingering File Handles (#71)
Fixing a bug where in various situations, the files used for temporary storage of query results would be leftover. In particular, the following changes were made:
* When the dispose query request is submitted, the corresponding query is now disposed in addition from being removed from the list of active queries
* When a query is cancelled, it is disposed after it is cancelled
* If a query already exists for a given ownerURI, the existing query is disposed before creating a new query
* All queries are disposed when the query execution service is disposed (ie, at shutdown of the service)

A unit test to verify the action of the dispose method for a ResultSet was added.

* Ensuring queries are disposed

Adding logic to dispose any queries when:
* URI that already has a query executes another query
* A request to dispose a query is submitted
* A request to cancel a query is submitted

* Small tweaks for cleanup of query execution service
2016-10-03 11:35:58 -07:00
Sharon Ravindran
20b64eadbf Feature/save selection (#64)
* Save selection

* Add tests

* Change filename in test

* Code cleanup

* Refactor handler

* Code cleanup

* Modify tests to have query selection

* Change variable declaration
2016-09-30 13:48:37 -07:00
Anthony Dresser
f22c8a7283 Feature/batch line info (#56)
* inital pipe of line numbers and getting text from workspace services

* tests compile

* Fixed bug regarding tests using connections on mac

* updated tests

* fixed workspace service and fixed tests

* integrated feedback
2016-09-22 17:58:45 -07:00
Benjamin Russell
93a75f1ff4 Format Cell Values (#62)
* WIP for ability to localize cell values

* Changing how DateTimeOffsets are stored, getting unit tests going

* Reworking BufferFileStreamWriter to use dictionary approach

* Plumbing the DbCellValue type the rest of the way through

* Removing unused components to simplify contract

* Cleanup and making sure byte[] appears in parity with SSMS

* CR comments, small tweaks for optimizing LINQ
2016-09-22 12:00:32 -07:00
Benjamin Russell
41198e9357 Adding sr.strings file and removing hard-coded strings (#52)
* Strings sweep for connection service

* String sweep for credentials service

* String sweep for hosting

* String sweep for query execution service

* String sweep for Workspace service

* Renaming utility namespace to match standards

Renaming Microsoft.SqlTools.EditorServices.Utility to
Microsoft.SqlTools.ServiceLayer.Utility to match the naming changes done a
while back. Also renaming them on the files that use them

* Namespace change on reliable connection

* Adding the new resx and designer files

* Final bug fixes for srgen

Fixing flakey moq package name

* Removing todo as per @kevcunnane

* Adding using statements as per @llali's comment

* Fixing issues from broken unit tests

Note: This feature contains changes that will break the contract for
saving as CSV and JSON. On success, null is returned as a message instead
of "Success". Changes will be made to the vscode component to handle this
change.
2016-09-16 16:18:25 -07:00