* Fix HDFS with AD auth for browse, read
- HDFS now fully supports expanding nodes for all levels, including using cookie for auth
- HDFS now support reading files from HDFS
- HDFS write file is broken and will be fixed (either in PR update or separate PR)
- Removed hack to use gateway-0 instead of actual DNS name now these are supported. Needed for testing
* Fix Jupyter error using new DMV with endpoints
Partially working AD support for BDCs with some known issues
- Plumbed through kerberos support to Notebooks.
- Using "gateway-0" for service temporarily as service endpoints API doesn't yet return correct DNS name. Will update in separate PR once available
- Plumbed kerberos auth to HDFS, Spark. Only partially working as we use same token on each call
- Will fix in separate PR, as this requires a refactor of WebHDFS library. Will need to either get new token every time or set a cookie, both of which require refactors
- Fixed error when Data Service node expansion failed and blocked all OE expansion
- Support for SqlToolsService change to use new cluster endpoints DMV
- Updated API to add new endpoints field to replace IP + port
- Added logic to handle case where endpoints for Yarn, Grafana etc. are in the list
- Sort list and use expected new localized strings
- Updated SqlToolsService to include support for new DMV
- Add "gateway-0" handling in Jupyter session as workaround for lack of domain names in endpoints list
* Replace Big Data Cluster with big data cluster
Official docs guidance is to use "big data cluster" instead of "Big Data Cluster"
* Use doublequotes and full product name
* Remove --user option when doing pip installs for our standalone Python version.
* Use force-reinstall option when installing sparkmagic since we use a custom version.
* Use force-reinstall when installing pip packages from Manage Packages dialog so that dependencies don't get split across multiple locations.
* Update PATH after install to include additional package directories.
Fixes#5231
- Add stdin handling. Has to be at UI level so add plumb through handling
- Add unit tests
- Add new StdIn component.
Testing:
Unit Tests and manual testing of following:
- Prompt for password using `getpass` in python.
- Password prompt is hidden since "password" is true.
- Hit enter, it completes
- prompt, stop cell running, StdIn disappears
- prompt, hit escape, stdIn disappears and stdIn request is handled.
Issues: focus isn't always set to the input even though we call focus.
Will investigate this further.
* Close jupyter and python
* Ensure we stop jupyter correctly on process end
* dont stopServer from clientSession shutdown
* PR comments
* close notebook after each test
This is a partial fix that lays groundwork for full "Prompt to connect" if a kernel needs a connection.
I am waiting on Yurong's refactoring of connection handling before doing any of the prompt work.
- Adds kernel metadata about whether a connection is required.
- For Jupyter, only Spark kernels are listed as requiring a connection
- If this is true and there's no active connection, will show notification and not call execute
In the future, this path will still be used if user is prompted to connect and cancels out.
The future change will be to inject a "connect" handler from notebook.component to the cell callback and use to set connection context
* Fix#4029 Ensure changeKernels always resolves, even in error states
This is necessary to unblock reverting the kernel on canceling Python install
- startSession now correctly sets up kernel information, since a kernel is loaded there.
- Remove call to change kernel on session initialize. This isn't needed due to refactor
- Handle kernel change failure by attempting to fall back to old kernel
- ExtensionHost $startNewSession now ensures errors are sent across the wire.
- Update AttachTo and Kernel dropdowns so they handle kernel being available. This is needed since other changes mean the session is likely ready before these get going
* Fix to handle python cancel and load existing scenarios
- Made changes to handle failure flow when Python dialog is canceled
- Made changes to handle initial load fail. Kernel and Attach To dropdowns show No Kernel / None and you can choose a kernel
- Added error wrapping in ext host so that string errors make it across and aren't lost.
* Fix#4356 New Notebook from connection doesn't connect
Fix new notebook error by passing profile instead of ID.
- I could've just sent the ID over, but this fix sets the stage for disconnected connections to work (since we have enough info to properly connect).
- There's a bug in NotebookModel blocking the disconnected connection part working, but Yurong's in progress fixes will unblock this. Hence checking in as-is and working to properly unblock once that's in.
* Support connection profile in commandline service
- Added new context API for things that want to work on commandline and object explorer
- Refactored commandlineservice slightly to be async & have a simpler execution flow (far fewer if/else statements)
* Fix unit tests
- Fixed 2 issues raised by tests (sholdn't do new query if no profile passed, shouldn't error on new query failing)
- Updated unit tests to pass as expected given changes to the APIs.
* Start single client session based on the default kernel or saved kernel in NB.
* Added kernel displayName to standardKernel.
Modified name to allign wtih Juptyer Kernel.name.
So we can show the displayName during startup and use the name to start the session.
* Change session.OnSessionReady event in KernelDropDown
* Added model.KernelChnaged for switching kernel in the same provider
* Fixed session.Ready sequence
* Fixed merge issues
* Solve merged issue
* Fixed wrong kernel name in saved NB
* Added new event in Model to notify kernel change.
Toolbar depends on ModelReady to load
* Change attachTo to wait for ModelReady like KenelDropDown
* sanitizeSavedKernelInfo to fix invalid kernel and display_name. For example: PySpark1111 and PySpark 1111
* Added _contextsChangingEmitter to change loadContext msg when changing kernel
* Resolve PR comments
- Jupyter completion item support was awaiting info before responding. Fix is to check if this is even a notebook cell first, then only await stuff if that's true
