* #4339: Kernel change event occurs after model load
* #4347: Code cleanup - Notebooks Save
* Remove save method from sqlops
* Adding save method to api's
* Adding save method to ext host
* Misc change
* Fix#4356 New Notebook from connection doesn't connect
Fix new notebook error by passing profile instead of ID.
- I could've just sent the ID over, but this fix sets the stage for disconnected connections to work (since we have enough info to properly connect).
- There's a bug in NotebookModel blocking the disconnected connection part working, but Yurong's in progress fixes will unblock this. Hence checking in as-is and working to properly unblock once that's in.
* Support connection profile in commandline service
- Added new context API for things that want to work on commandline and object explorer
- Refactored commandlineservice slightly to be async & have a simpler execution flow (far fewer if/else statements)
* Fix unit tests
- Fixed 2 issues raised by tests (sholdn't do new query if no profile passed, shouldn't error on new query failing)
- Updated unit tests to pass as expected given changes to the APIs.
* Start single client session based on the default kernel or saved kernel in NB.
* Added kernel displayName to standardKernel.
Modified name to allign wtih Juptyer Kernel.name.
So we can show the displayName during startup and use the name to start the session.
* Change session.OnSessionReady event in KernelDropDown
* Added model.KernelChnaged for switching kernel in the same provider
* Fixed session.Ready sequence
* Fixed merge issues
* Solve merged issue
* Fixed wrong kernel name in saved NB
* Added new event in Model to notify kernel change.
Toolbar depends on ModelReady to load
* Change attachTo to wait for ModelReady like KenelDropDown
* sanitizeSavedKernelInfo to fix invalid kernel and display_name. For example: PySpark1111 and PySpark 1111
* Added _contextsChangingEmitter to change loadContext msg when changing kernel
* Resolve PR comments
* Move to using notebook language by default, with override in cell
* Update cell language on kernel change
* Tweak language logic so that it prefers code mirror mode, then falls back since this was failing some notebooks
* Add new package.json contribution to define language magics. These result in cell language changing. Language is cleared out on removing the language magic
* Added support for executing Python, R and Java in the SQL Kernel to prove this out. It converts to the sp_execute_external_script format
TODO in future PR:
* Need to hook up completion item support for magics (issue #4078)
* Should add indicator at the bottom of a cell when an alternate language has been detected (issue #4079)
* On executing Python, R or Java, should add some output showing the generated code (issue #4080)
* working on formatting
* fixed basic lint errors; starting moving things to their appropriate location
* formatting
* update tslint to match the version of vscode we have
* remove unused code
* work in progress fixing layering
* formatting
* moved connection management service to platform
* formatting
* add missing file
* moving more servies
* formatting
* moving more services
* formatting
* wip
* moving more services
* formatting
* move css file
* add missing svgs
* moved the rest of services
* formatting
* changing around some references
* formatting
* revert tslint
* revert some changes that brake things
* formatting
* fix tests
* fix testzx
* fix tests
* fix tests
* fix compile issue
* Fix error where kernel name was compared to itself. This doesn't break anything right now since we happen to have special handling of Python3, and for other kernels they share the same set of supported providers (which is what the check is used for). However it needs fixing for next release.
* Fix console error due to queryTextEditor trying to access model before it's ready
* Fix issues where notebook model hooked to session events multiple times
* Removed calls that weren't needed such as loadActiveContexts (if undefined, does nothing) and passing connection to initialize method
* Fixed#3954
The problem is: connectionProfileId is not passed into New Notebook flow.
The fix is: plumbing connectionProfileId via NotebookInput.
* Resolved PR comments
* added initial data explorer viewlet
* added dataexplorer contribution point
* removed test view
* remove unused imports
* added flag and tests
* CR comments
* added icon for data explorer
- Editor layout gets called sometimes when other events happen (and Notebook isn't visible)
- Add in a layout call on re-setting input so the cell is updated. This fixes the problem by laying out once the UI is visible again.
Note: long term, should really be destroying the UI (while preserving the model), then restoring it including scroll selection etc. and hooking back up to the model. That is... much more work, but something we'll need long term to avoid issues where we have many Notebooks open at once. Not in scope for this PR
- Added `runCell` API. Updated runCell button to listen to events on the model so it'll reflect run cell when called from other sources
- Plumbed through kernelspec info to the extension side so when changed, it's updated
- Fixed bug in ConnectionProfile where it didn't copy from options but instead overrode with empty wrapper functions
Here's the rough test code (it's in the sql-vnext extension and will be out in a separate PR)
```ts
it('Should connect to local notebook server with result 2', async function() {
this.timeout(60000);
let pythonNotebook = Object.assign({}, expectedNotebookContent, { metadata: { kernelspec: { name: "python3", display_name: "Python 3" }}});
let uri = writeNotebookToFile(pythonNotebook);
await ensureJupyterInstalled();
let notebook = await sqlops.nb.showNotebookDocument(uri);
should(notebook.document.cells).have.length(1);
let ran = await notebook.runCell(notebook.document.cells[0]);
should(ran).be.true('Notebook runCell failed');
let cellOutputs = notebook.document.cells[0].contents.outputs;
should(cellOutputs).have.length(1);
let result = (<sqlops.nb.IExecuteResult>cellOutputs[0]).data['text/plain'];
should(result).equal('2');
try {
// TODO support closing the editor. Right now this prompts and there's no override for this. Need to fix in core
// Close the editor using the recommended vscode API
//await vscode.commands.executeCommand('workbench.action.closeActiveEditor');
}
catch (e) {}
});
it('Should connect to remote spark server with result 2', async function() {
this.timeout(240000);
let uri = writeNotebookToFile(expectedNotebookContent);
await ensureJupyterInstalled();
// Given a connection to a server exists
let connectionId = await connectToSparkIntegrationServer();
// When I open a Spark notebook and run the cell
let notebook = await sqlops.nb.showNotebookDocument(uri, {
connectionId: connectionId
});
should(notebook.document.cells).have.length(1);
let ran = await notebook.runCell(notebook.document.cells[0]);
should(ran).be.true('Notebook runCell failed');
// Then I expect to get the output result of 1+1, executed remotely against the Spark endpoint
let cellOutputs = notebook.document.cells[0].contents.outputs;
should(cellOutputs).have.length(4);
let sparkResult = (<sqlops.nb.IStreamResult>cellOutputs[3]).text;
should(sparkResult).equal('2');
try {
// TODO support closing the editor. Right now this prompts and there's no override for this. Need to fix in core
// Close the editor using the recommended vscode API
//await vscode.commands.executeCommand('workbench.action.closeActiveEditor');
}
catch (e) {}
});
});
```
* Added Unified connection support
* Use generic way to do expandNode.
Cleanup the ported code and removed unreference code. Added as needed later.
Resolved PR comments.
* Minor fixes and removed timer for all expanders for now. If any providers can't response, the tree node will spin and wait. We may improve later.
* Change handSessionClose to not thenable.
Added a node to OE to show error message instead of reject. So we could show partial expanded result if get any.
Resolve PR comments
* Minor fixes of PR comments
* Scenarios work besides loading saved kernel
* Fix compilation issue
* Save and load functional
* Fix loading kernesl issue when sql kernel is not enabled
* Fix language mapping to not be hardcoded any longer
* Remove unnecessary comment
* PR Comments vol. 1
* Code cleanup, use ConnectionProfile instead of IConnectionProfile when accessing serverName
* PR changes vol. 2
* One final comment for PR
* Fix linting issue
* working on formatting
* fixed basic lint errors; starting moving things to their appropriate location
* formatting
* update tslint to match the version of vscode we have
* remove unused code
* work in progress fixing layering
* formatting
* moved connection management service to platform
* formatting
* add missing file
* moving more servies
* formatting
* moving more services
* formatting
* wip
* moving more services
* formatting
* revert back tslint rules
* move css file
* add missing svgs
* add an extension for integration tests
* setup ads before running test
* test setup
* test cases
* bash script
* shorter temp folder name
* code cleanup
* add commented out original code
* fix test error
* test result path
* rename results file
* change file path
* report smoke test results
* test stablize
* test stablization and configurable test servers
* fix smoke test error
* connection provider
* simplify the integration test script
* add comment
* fix tslint error
* address PR comments
* add temp log to check whether the environment variable is already set
* remove temp log
* move api definition to testapi typing file
* exclude integration tests extension
* address comments
* working on options dialog
* working through options dialog
* trying to work through modifying options dialog
* working on converting scrollablesplitview
* fixed options working through profiler
* fix profiler
* fix account dialog
* trying to fix problems with splitpanel
* fix insights dialog
* moving through
* fix last list, need to verify looks and functionality
* fix look of account dialog
* formatting
* formatting
* working through scrollable bugs
* working on problem with view size
* fix margin issues
* fix styler for dialogs
* add panel styles to insights
* create instantiation issues
* fix test
* fix test
* remove unused code
* formatting
* working through insight dialog issues
* fix table updating
* remove console logs
* Placeholder cell to add new real cells
* Fix links in notebooks to show correct color, rely on angular ngif for placeholder
* Fix failing test where one cell was expected by default
* Remove unnecessary TODO
* First crack tsql notebook (no output rendered yet)
* getting messages back
* intellisense working first cell, no connection errors
* sql notebook cell output functioning
* Latest SQL noteobook changes
* Undo change to launch.json
* Plumbing providers through
* Kernels shown from multiple providers, can switch between them. No mementos yet
* Ensure we have a feature flag for SQL notebooks, ensure existing functionality still works
* Fix tslint duplicate imports issue
* Addressing PR comments
* second round of PR feedback to cleanup notebook service manager code
* merge latest from master
* Refactor JSON and format files to model and fix tabs -> spaces issues
This is in prep for some work to reuse these code files inside the model,
so pushing as its own PR to keep the next piece of work clean.
* Fix#3680 Notebooks: outputs with string arrays rendered incorrectly
- Add support for processing v4 format files loaded from disk
- Prep support for v3 notebooks by adding placeholder code for that
* Fix failing tests and add specific one for this bug
* Remove references to v5
- Set markdown as language for markdown cell
- Fix issue where after loading language from cell metadata, always override it. This made the "language" feature irrelevant in the cell.
- Fixed tests with new behavior (assumption: cell-level language overrides notebook-level definition) and added new test to cover this too
- Add edit API that can be used in the extension
- Separated document and editor classes out since this is the point those get big. I can refactor back in if needed to ease code review
- Based this off text editing APIs but tweaked for the fact this is a cell/array based set of edits
* agent feature usage metrics
* generalized feature telemetry via dialogs
* renamed eventName property to dialogName
* made dialogName an optional field
initial support for Notebook extensibility. Fixes#3148 , Fixes#3382.
## Design notes
The extensibility patterns are modeled after the VSCode Document and Editor APIs but need to be different since core editor concepts are different - for example Notebooks have cells, and cells have contents rather than editors which have text lines.
Most importantly, a lot of the code is based on the MainThreadDocumentsAndEditors class, with some related classes (the MainThreadDocuments, and MainThreadEditors) brought in too. Given our current limitations I felt moving to add 3 full sets of extension host API classes was overkill so am currently using one. Will see if we need to change this in the future based on what we add in the additional APIs
## Limitations
The current implementation is limited to visible editors, rather than all documents in the workspace. We are not following the `openDocument` -> `showDocument` pattern, but instead just supporting `showDocument` directly.
## Changes in this PR
- Renamed existing APIs to make clear that they were about notebook contents, not about notebook behavior
- Added new APIs for querying notebook documents and editors
- Added new API for opening a notebook
- Moved `New Notebook` command to an extension, and added an `Open Notebook` command too
- Moved notebook feature flag to the extension
## Not covered in this PR
- Need to actually implement support for defining the provider and connection IDs for a notebook. this will be important to support New Notebook from a big data connection in Object Explorer
- Need to add APIs for adding cells, to support
- Need to implement the metadata for getting full notebook contents. I've only implemented to key APIs needed to make this all work.
