Update the ScriptingService to expose new scripting JSON-RPC APIs that use the SqlScriptPublishModel for script generation.
The SqlScriptPublishModel is the model behind the SSMS scripting wizard. To enable scripting for CLI tools, we've ported SqlScriptPublishModel to .NET Core. The SqlScriptPublishModel wraps the SMO scripting APIs for .sql script generation.
1) Added three new requests to the ScriptingService: ScriptingRequest, ScriptingListObjectsRequest, ScriptingCancelRequest.
2) Generating scripts are long running operations, so the ScriptingRequest and ScriptingListObjectsRequest kick off a long running scripting task and return immediately.
3) Long running scripting task reports progress and completion, and can be cancelled by a ScriptingCancelRequest request.
4) Bumped the SMO nuget package to 140.17049.0. This new version contains a signed SSMS_Rel build of SMO with the SqlScriptPublishModel.
5) For testing, adding the Northwind database schema
TODO (in later pull requests)
1) Integrate the new ScriptingService APIs with the ConnectionService
2) Integrate with the metadata support recently added
* added more batch parser tests
* fixed merge conflicts
* change debug type to portable
* fixed imports according to review
* fixed sql cmd tests
* making change to check appveyor tests
* removed coverage result file
* Fix the OE service where retuns database as the root node if the connection contains database name
* Fix OE tests
* addressed the comments
* addresses comment
* fix OE test and add more tests
* fix VerifyAdventureWorksDatabaseObjects test
Update to include the Row ID with a DbCellValue. This will be super useful to us for the purposes of managing individual rows via slick grid.
Although this changes the API for edit/subset and query/subset, it is effectively backwards compatible since it's adding a parameter, not removing anything.
* New DbCellValue has an internal Row ID
* Adding unit tests
* Connect with different properties should actually change context
- Up to now, calling Connect for a previously-connected URI would disconnect, then reconnect ot the original (not new) target. WIth these changes we handle changes to database name or other key properties by updating the ConnectionInfo and connecting to the new target
- Some interesting scenarios are raised by our API, notably that an empty database name maps to the default DB (which we know nothing about). This limits the new feature such that only if the DB Name is specified, we'll change the connection. Hence 2 calls to an empty DB will not result in a DB change.
Additional changes:
- After discussion with Ben, we're simplifying the cancellation logic. He had made changes to support this, so the main update is that we dispose the token in the final block after its last use (hence avoiding a disposed exception) and clean up the number of Waits required since we already have async cancellation support
- Factored some logic such that the OnConnection callback isn't invoked until after we've updated the database name in the GetConnectionCompleteParams method. Again, this supports reporting the actual DB name instead of leaving it blank for default DB requests.
* PR comment fixes
This change ensures that when calling `requestContext.SendError` you are only able to supply parameters that match the language service beta protocol expected Error object. In other words, you have to provide an error message and optionally and error code.
# **BREAKING API CHANGES**
This will break displaying errors in Microsoft/vscode-mssql. I will be making changes to properly handle the error object shortly.
* Adding contract for returning Error objects as per LanguageService "protocol"
* Fixes throughout codebase to send only error message in error cases
Cleanup of CredentialServiceTest unit test class
Adding standard error handling for event flow validator
* Adding optional data field as per protocol spec
https://github.com/Microsoft/language-server-protocol/blob/master/protocol.md
* Adding optional validation for error objects
Small change that removes logging complete JSON objects that are sent back from the tools service via JSON RPC. Now the message type and method is logged only. In previous releases, if debug logging was enabled, all JSON objects sent from the tools service would be logged in the debug log, if enabled. This was bad for 2 reasons:
1) sensitive data (passwords from credential service, cell contents) would be logged
2) the log would quickly become massive due to all rows being written to the log, every time they were requested.
This change enhances the way that edit/updateCell and edit/revertCell operations are performed.
## **THE API BREAKING CHANGES**:
* edit/updateCell now returns an EditCell (a DbCellValue with a dirty flag) and a row dirty flag.
* edit/revertCell now returns an EditCell (a DbCellValue with a dirty flag) and a row dirty flag.
If by setting the value of a cell via edit/updateCell the row no longer has any edits (an "implicit revert"), the entire row's edit will be removed from the cache. Additionally, if by requesting edit/revert all the pending edits for a row are removed, the entire row's edit will be removed from the cache. This will prevent issues where committing will generate an invalid script because it has no pending changes.
* Adding EditCell class
Returning EditCell with EditUpdateCellResult
* Adding code that will remove a row update if the row is clean after a cell update
* Adding code that will return an EditCell and row dirty flag when a cell is reverted.
If the row is reverted by the cell revert, the pending update will be removed
* Comments for edit cell
* Changes as per pull request comments
This change adds a couple things
_Multipart Identifier Decoding_
The ability to decode a multipart identifier (with or without escaping) has been added to the SqlScriptFormatter utility class. This code is utilized to split a table name provided to the edit/initialize request into schema and table name.
_Default Schema Workaround_
The code that retrieves the SMO metadata objects originally used the `[]` operator to access the objects. Due to a bug(?) in SMO, this results in problems when loading tables without a default schema (in our case if you're logged in as SA). Using the metadata object constructors gets around this issue, we are explicitly using them.
* Adding decoding of multipart identifiers
Adding code fix for default schema issue
* Adding some more localizable strings for errors when loading metadata
* Adding localization files... again?
* Changes as per pull request comments
* Ensures that metadata is "extended" before creating a new row edit object
* Adds the commit handler to edit data service initialization calls
* Unit tests for the associated changes
* fixed peek definition tokenizing
* fixed signature help test
* added new heuristic for PeekDefinition behaviour
* added tests for new heuristic
* fixed code according to Kevin's CR
* fixed failing test due to shared connection
* changed uri for procedure test
* Major refactor of EditDataMetadata providers
* EditMetadataFactory generates "basic" EditTableMetadata objects based entirely on SMO metadata
* SmoEditTableMetadata no longer depends on SMO, making it unecessary to mock it
* Renamed SmoEditTableMetadata to EditTableMetadata
* EditTableMetadata can be extended with DbColumnWrappers
* Moving logic for extending a EditColumnMetadata into that class
* I *think* this will work for async execution of initialize tasks
* Fixing unit tests for new Edit(Table|Column)Metadata classes
* Async stuff that works! And passes unit tests
* Adding unit tests
Adding .idea to gitignore
* Adding message to the EditSessionReadyEvent
* Fixes from dev merge
* Fixing unit tests that Rider didn't catch as failing
May have been a bit heavy-handed with the async/await stuff
* Couple changes as per PR comments
* Changing query/subset API to only use Result on success, Error on error
* Creating an interservice API for getting query result subsets
* Updates to subset API
* RowStartIndex is now long
* Output of query/subset is a 2D array of DbCellValue
* Adding LongSkip method to LongList to allow skipping ahead by longs
* Moving LongList back to ServiceLayer utilities. Move refactoring
* Stubbing out request for edit/subset
* Initial implementation of getting edit rows
* Unit tests for RowEdit and RowDelete .GetEditRow
* Fixing major bugs in LongList implementation, adding much more thorough tests
* Adding some more unit tests and fixes to make unit tests pass
* Fixing comment
* Initial scripting commit
* Fix the script CREATE integration test
* Use language service binding queue for scripting
* Update scripting service to have methods for each operation
* Add scripter class
* Fix build break.
* Fix a few bugs
* Use scripter class instead of PeekDefinition class.
* Fix scripting test break
* Fix header incorrectly saying file is generated.
* Add localization tests to address a large block of code not covered by automation.
* Moving system usings to top of using list
* Bump SMO bits to align with 17.0 RC3 release
* Add updated SMO binary
* Fix formatter failures caused by SqlProc breaking changes
- Changes to SqlProcedureDefinition subclasses mean that the definition starts at "Create Procedure" instead of after it. This highlighted an issue where the prefix region handling was via comma separated list (which adds newlines to the prefix region) instead of newline separated list (which does not).
- Changed the structure to use the 2 different formatters for relevant regions, and verified all tests now pass.
- Also removed unnnecessary Export clause from the SqlTableDefinitionFormatter, since this was spotted during debugging
* Fix sample project.json dependencies
- Fix up the dependencies in our samples as this was failing the build on TravisCI. Should consider in the future whether or not we want to exclude these from CI runs, but for now the benefit is that we're forced to keep the samples up to date.
* Fix dispose pattern usage in SaveAsWriterBase
* Add SaveAsExcel feature
This adds the save as excel function to the backend. To reduce large dependency and run on dotnet core now, this implementation use a raw excel writer (the SaveAsExcelFileStreamWriterHelper.cs) instrad of popular excel library, such as EPPlus or OpenXmlSdk.
* Fix can not open the generated excel file in google sheet
For the file name inside excel, google uses a case sensitive path while
Excel doesn't. This change fix the case, so that the file name matches
the one in x1/_rels/workbook.xml.rels
* Fix datetime doesn't recognized by google sheet
Google doesn't support cell type t="d" with ISO 8601 date. (From stackoverflow thread and testing), thus use the old way of excel datetime, which uses double to present datetime
* update to use xmlwriter
* Add basic unit tests for SaveAsExcelFileStreamWriterHelper
* refactor: simplify the public interface of the SaveAsExcelFileStreamWriterHelper
* update private fields names based on the name convention
* Add comments to classes of SaveAsExcel feature
* clean up SaveAsExcelFileStreamWriterHelper
- change SaveAsExcelFileStreamWriterHelper from public to internal
- remove the PenddingRowEndTag function from referenceManager
- change the SaveAsExcelFileStreamWriterHelper(stream) to default leaveOpen to false to match the normal behavior
- change the rowreference to use XmlConvert to convert int to string
- rename writeSetting to writerSetting and add private
* fix CI test error for SaveAsExcel
* remove ExporterException in SaveAsExcel
* fix lefe over CSV to Excel in the comments
* refactor to be consistent with JsonWriter and remove the comment
* remove commented out test
The test is too slow to run
* fix typo in comment
* refactor SaveAsExcel to the coding standard
* refactor rewrite the WriteStyle with XmlWriter
* Add licence header
* reverse mistakenly checked-in changes
* fix: left-over CSV in commets
* remove duplicate check
The check was done in the IncreaseColumnReference, but that check is too late in case of too many columns. All the addCell do the check at the begining now
* fix TimeSpan more than 24 hours
* fix AddRowMustBeCalledBeforeAddCellException test
This is due to remove duplicate call to AssureColumnReference in WriteAndIncreaseColumnReference
* fix: TimeSpan will write twice
* style: change retun in the switch to break
* Add bool format
* remove todo in comment
This provides extra safeguard in the cost of one memory access when null.
- Ensure minimum of 2 newlines after GO statement
- All existing unit tests are passing, indicating this is respecting scenarios where users have comment lines after GO statements (this happens in many other tests)
* Initial metadata service and scripting service files
* Simple metadata lookup with SMO objects
* Add metadata type class
* Remove SMO from metadata service.
* Cleanup metadata service SQL
* Initial MetadataService test
* Add scripting commands
* Add metadata test case
* Remove sleep used for testing
* Use random table name in metadata test
* Add scripting tests
The SqlScriptPublishModel is the model behind the SSMS scripting wizard. It has been ported to .NET Core to enable scripting scenarios to SqlToolsService. This change adds the SqlScriptPublishModel nuget package to the checked in packages in the bin\nuget folder.
Very small feature add to make sure that values entered for `TIME` columns are < 24hrs. If the value is >=24hrs when setting the column, an exception will be thrown that the edit/updateCell handler will catch and convert into an error on the JSON RPC side.
* Adding validation of timespan columns
* Fixing a merge conflict
