* Reopen connections prior to creating query execution data readers
* Reopen connection if an exception was rasied executing the query
* Fix unit tests
This change modifies the logging framework within sqltoolservice.
Moves away from custom Logger object to start using .Net tracing framework. It supports for the static Trace and TraceSource way of logging. For all new code it is recommend that we log the log messages using the existing static Logger class, while the code changes will continue to route the older Trace.Write* calls from the process to same log listeners (and thus the log targets) as used by the Logger class. Thus tracing in SMO code that uses Trace.Write* methods gets routed to the same file as the messages from rest of SQLTools Service code.
Make changes to start using .Net Frameworks codebase for all logging to unify our logging story.
Allows parameter to set tracingLevel filters that controls what kinds of message make it to the log file.
Allows a parameter to set a specific log file name so if these gets set by external code (the UI code using the tools service for example) then the external code is aware of the current log file in use.
Adding unittests to test out the existing and improved logging capabilities.
Sequences of checkins in development branch:
* Saving v1 of logging to prepare for code review. Minor cleanup and some end to end testing still remains
* Removing local launchSettings.json files
* added support for lazy listener to sqltoolsloglistener and removed incorrect changes to comments across files in previous checkin
* Converting time to local time when writing entries to the log
* move the hosting.v2 to new .net based logging code
* removing *.dgml files and addding them to .gitignore
* fixing typo of defaultTraceSource
* Addressing pull request feedback
* Adding a test to verify logging from SMO codebase
* propogating changes to v1 sqltools.hosting commandoptions.cs to the v2 version
* Fixing comments on start and stop callstack methods and whitespaces
* Commenting a test that got uncommented by mistake
* addding .gitattributes file as .sql file was observed to be misconstrued as a binary file
* fixed bug where execution time was wrong for multiple batches
* fixed bug where multiple execution would show wrong execution time
* simplified logic
* Support `GO N` syntax to execute multiple times
- Plumbed through the batch execution count from the parser and used in the batch execution code path
- Functionality matches SSMS:
- Outputs loop start/end messages that match SSMS if you're doing multi-batch execution
- Outputs an "ignoring failure" error if an error happens during a batch
- Added tests for this
- Manually verified end to end also
* Fixing test error
- HandleSimpleExecuteRequest now handles the case where no rows are in a result by cleanly returning a success message but with no rows included. This is handled in the front-end instead and goes through the standard path (with a clean explanation message) instead of showing a `error: no results to return`
- MetadataService was always meant to include the type name in the return result, as otherwise the front end has to guess. In order to fix a bug where this resulted in scripting based on the metadata failing (as front-end used `Procedure` instead of `StoredProcedure`), I'm returning the data here. I'll have a matching front end fix but this is overall a good solution to have.
* Fix to make sql exceptions surface properly to user (with important notes!)
* Adding support for detecting column size issues when updating a cell
* Adding unit tests for the exception on read scenario
* Replacing Dictionary with ConcurrentDictionary since values are accessed in async contexts
* Adding new method to allow async tasks to be executed in the exception continuation
* Adding unit tests for the aforementioned
* Adding exception handling to async tasks in file browser service
* Updating query execution async handling to use the async version
* Removing unnecesary send result from continuewithonfaulted
* WIP
* This code makes it work!
* Adding similar exception hanling behavior to saving result sets
* Adding unit tests for new extension methods
Auto-cleanup of proj file whitespace
* Implementing changes as per code review comments
* Fixing issue where Guids weren't being stored in the service buffer file as Guids
* Removing unnecessary ReadSqlGuid method
* Adding unit test for revert cell integration
* Adding unit test for edit session initialization
* Revert "Adding unit test for revert cell integration"
This reverts commit a949926f2ebbb0c39f776edba76d999af4f3f3e9.
* Revert "Adding unit test for edit session initialization"
This reverts commit ff9e5a7d0e5392b27257e57db64879699d73d21c.
* Initial changes to have serialization generate its own .exe
* Removed additional project from sln file
* remove all references to removed temporary project
* Moved shared contracts into own dll and fixed imports. Addressed PR comments
* Undid having a separate contracts project since that'll be a task for later on. Moved dbcellvalue and saveresultsrequest to Hosting, where they will be imported and shared by the service layer and serialization projects
* Switched backslashes in project reference in csproj file to forward slashes for consistency
* Moved necessary contracts back to service layer. Refactored CommandOptions to reduce code duplication. Addressed miscellaneous PR suggestions
* Accidentally left these files out of previous commit
* Initialized loggers for serialization and credentials with the logging directory provided by the cmd line arg, if there is one
* Changed default log directory paths for serialization and credentials. Removed unnecessary cast and added a copyright
* Changed name of generated executable for serialization service
* removed unnecessary object cast
* removing unnecessary imports and addressing other PR comments
* change simple execute to open a new connection and close it every query
* updated tests for simple execute
* removed an unnecessary connect
* refactored code to be more readable
* global try catch on simple execute
* added multiple execution test
* update execution to be asynchrous; update tests to account for asynchrounous nature
- Additional handling of document events with "perforce:" or other SCM strings
- Much of the handling had been added already, but adding in additional validation and ensuring that everywhere `Workspace.GetFile` is called we add a not-null check to avoid null reference errors
* inital request
* refactored query execution failure callback to take exception
* added failure callback to execute and return
* added test for query execute and return
* updated params
* removed dead code
* addressed feedback; added multiple active result set support; updated tests
* addessed feedback and added testing and errors and verification
* change <= to ==
* changed name of trashQ to removedQuery
Update to include the Row ID with a DbCellValue. This will be super useful to us for the purposes of managing individual rows via slick grid.
Although this changes the API for edit/subset and query/subset, it is effectively backwards compatible since it's adding a parameter, not removing anything.
* New DbCellValue has an internal Row ID
* Adding unit tests
This change ensures that when calling `requestContext.SendError` you are only able to supply parameters that match the language service beta protocol expected Error object. In other words, you have to provide an error message and optionally and error code.
# **BREAKING API CHANGES**
This will break displaying errors in Microsoft/vscode-mssql. I will be making changes to properly handle the error object shortly.
* Adding contract for returning Error objects as per LanguageService "protocol"
* Fixes throughout codebase to send only error message in error cases
Cleanup of CredentialServiceTest unit test class
Adding standard error handling for event flow validator
* Adding optional data field as per protocol spec
https://github.com/Microsoft/language-server-protocol/blob/master/protocol.md
* Adding optional validation for error objects
* Changing query/subset API to only use Result on success, Error on error
* Creating an interservice API for getting query result subsets
* Updates to subset API
* RowStartIndex is now long
* Output of query/subset is a 2D array of DbCellValue
* Adding LongSkip method to LongList to allow skipping ahead by longs
* Moving LongList back to ServiceLayer utilities. Move refactoring
* Stubbing out request for edit/subset
* Initial implementation of getting edit rows
* Unit tests for RowEdit and RowDelete .GetEditRow
* Fixing major bugs in LongList implementation, adding much more thorough tests
* Adding some more unit tests and fixes to make unit tests pass
* Fixing comment
* Fix dispose pattern usage in SaveAsWriterBase
* Add SaveAsExcel feature
This adds the save as excel function to the backend. To reduce large dependency and run on dotnet core now, this implementation use a raw excel writer (the SaveAsExcelFileStreamWriterHelper.cs) instrad of popular excel library, such as EPPlus or OpenXmlSdk.
* Fix can not open the generated excel file in google sheet
For the file name inside excel, google uses a case sensitive path while
Excel doesn't. This change fix the case, so that the file name matches
the one in x1/_rels/workbook.xml.rels
* Fix datetime doesn't recognized by google sheet
Google doesn't support cell type t="d" with ISO 8601 date. (From stackoverflow thread and testing), thus use the old way of excel datetime, which uses double to present datetime
* update to use xmlwriter
* Add basic unit tests for SaveAsExcelFileStreamWriterHelper
* refactor: simplify the public interface of the SaveAsExcelFileStreamWriterHelper
* update private fields names based on the name convention
* Add comments to classes of SaveAsExcel feature
* clean up SaveAsExcelFileStreamWriterHelper
- change SaveAsExcelFileStreamWriterHelper from public to internal
- remove the PenddingRowEndTag function from referenceManager
- change the SaveAsExcelFileStreamWriterHelper(stream) to default leaveOpen to false to match the normal behavior
- change the rowreference to use XmlConvert to convert int to string
- rename writeSetting to writerSetting and add private
* fix CI test error for SaveAsExcel
* remove ExporterException in SaveAsExcel
* fix lefe over CSV to Excel in the comments
* refactor to be consistent with JsonWriter and remove the comment
* remove commented out test
The test is too slow to run
* fix typo in comment
* refactor SaveAsExcel to the coding standard
* refactor rewrite the WriteStyle with XmlWriter
* Add licence header
* reverse mistakenly checked-in changes
* fix: left-over CSV in commets
* remove duplicate check
The check was done in the IncreaseColumnReference, but that check is too late in case of too many columns. All the addCell do the check at the begining now
* fix TimeSpan more than 24 hours
* fix AddRowMustBeCalledBeforeAddCellException test
This is due to remove duplicate call to AssureColumnReference in WriteAndIncreaseColumnReference
* fix: TimeSpan will write twice
* style: change retun in the switch to break
* Add bool format
* remove todo in comment
This provides extra safeguard in the cost of one memory access when null.
The main goal of this feature is to enable a command that will
1) Generate a parameterized command for each edit that is in the session
2) Execute that command against the server
3) Update the cached results of the table/view that's being edited with the committed changes (including computed/identity columns)
There's some secret sauce in here where I cheated around worrying about gaps in the updated results. This was accomplished by implementing an IComparable for row edit objects that ensures deletes are the *last* actions to occur and that they occur from the bottom of the list up (highest row ID to lowest). Thus, all other actions that are dependent on the row ID are performed first, then the largest row ID is deleted, then next largest, etc. Nevertheless, by the end of a commit the associated ResultSet is still the source of truth. It is expected that the results grid will need updating once changes are committed.
Also worth noting, although this pull request supports a "many edits, one commit" approach, it will work just fine for a "one edit, one commit" approach.
* WIP
* Adding basic commit support. Deletions work!
* Nailing down the commit logic, insert commits work!
* Updates work!
* Fixing bug in DbColumnWrapper IsReadOnly setting
* Comments
* ResultSet unit tests, fixing issue with seeking in mock writers
* Unit tests for RowCreate commands
* Unit tests for RowDelete
* RowUpdate unit tests
* Session and edit base tests
* Fixing broken unit tests
* Moving constants to constants file
* Addressing code review feedback
* Fixes from merge issues, string consts
* Removing ad-hoc code
* fixing as per @abist requests
* Fixing a couple more issues
This is a very large change. I'll try to outline what's going on.
1. This adds the **EditDataService** which manages editing **Sessions**.
1. Each session has a **ResultSet** (from the QueryExecutionService) which has the rows of the table and basic metadata about the columns
2. Each session also has an **IEditTableMetadata** implementation which is derived from SMO metadata which provides more in-depth and trustworthy data about the table than SqlClient alone can.
3. Each session holds a list of **RowEditBase** abstract class implementations
1. **RowUpdate** - Update cells in a row (generates `UPDATE` statement)
2. **RowDelete** - Delete an entire row (generates `DELETE` statement)
3. **RowCreate** - Add a new row (generates `INSERT INTO` statement)
4. Row edits have a collection of **CellUpdates** that hold updates for individual cells (except for RowDelete)
1. Cell updates are generated from text
5. RowEditBase offers some baseline functionality
1. Generation of `WHERE` clauses (which can be parameterized)
2. Validation of whether a column can be updated
2. New API Actions
1. edit/initialize - Queries for the contents of a table/view, builds SMO metadata, sets up a session
2. edit/createRow - Adds a new RowCreate to the Session
3. edit/deleteRow - Adds a new RowDelete to the Session
4. edit/updateCell - Adds a CellUpdate to a RowCreate or RowUpdate in the Session
5. edit/revertRow - Removes a RowCreate, RowDelete, or RowUpdate from the Session
6. edit/script - Generates a script for the changes in the Session and stores to disk
7. edit/dispose - Removes a Session and releases the query
3. Smaller updates (unit test mock improvements, tweaks to query execution service)
**There are more updates planned -- this is just to get eyeballs on the main body of code**
* Initial stubs for edit data service
* Stubbing out update management code
* Adding rudimentary dispose request
* More stubbing out of update row code
* Adding complete edit command contracts, stubbing out request handlers
* Adding basic implementation of get script
* More in progress work to implement base of row edits
* More in progress work to implement base of row edits
* Adding string => object conversion logic and various cleanup
* Adding a formatter for using values in scripts
* Splitting IMessageSender into IEventSender and IRequestSender
* Adding inter-service method for executing queries
* Adding inter-service method for disposing of a query
* Changing edit contract to include the object to edit
* Fully fleshing out edit session initialization
* Generation of delete scripts is working
* Adding scripter for update statements
* Adding scripting functionality for INSERT statements
* Insert, Update, and Delete all working with SMO metadata
* Polishing for SqlScriptFormatter
* Unit tests and reworked byte[] conversion
* Replacing the awful and inflexible Dictionary<string, string>[][] with a much better test data set class
* Fixing syntax error in generated UPDATE statements
* Adding unit tests for RowCreate
* Adding tests for the row edit base class
* Adding row delete tests
* Adding RowUpdate tests, validation for number of key columns
* Adding tests for the unit class
* Adding get script tests for the session
* Service integration tests, except initialization tests
* Service integration tests, except initialization tests
* Adding messages to sr.strings
* Adding messages to sr.strings
* Fixing broken unit tests
* Adding factory pattern for SMO metadata provider
* Copyright and other comments
* Addressing first round of comments
* Refactoring EditDataService to have a single method for handling
session-dependent operations
* Refactoring Edit Data contracts to inherit from a Session and Row
operation params base class
* Copyright additions
* Small tweak to strings
* Updated unit tests to test the refactors
* More revisions as per pull request comments
Adding a new setting to query execution setting that will change the display value we generate for `BIT` columns. The new setting is `DefaultDisplayBitAsNumber`. If true, bit columns will be displayed as 1 or 0. If false, they'll be displayed as true or false. The default value is true, to keep parity with SSMS behavior.
Enables us to solve https://github.com/Microsoft/vscode-mssql/issues/690 and https://github.com/Microsoft/vscode-mssql/issues/513
