Commit Graph

160 Commits

Author SHA1 Message Date
Lucy Zhang
b8435a67df Add telemetry for Python on Jupyter server startup (#19970)
* add telemetry for python on jupyter server startup

* remove extra lines
2022-07-11 12:20:22 -07:00
Chris LaFreniere
af3d307d40 Restart kernel initial implementation (#18835)
* Restart kernel initial implementation

* Update notebook extension TestKernel

* PR comments
2022-03-29 12:56:52 -07:00
Cory Rivera
ffdefd3b52 Add notebook extension support for .NET Interactive. (#18334)
* Also updated kernel dropdown to only include SQL aliased kernels when using SQL notebook provider.
2022-02-25 11:58:59 -08:00
Monica Gupta
8685137c27 TSGops -image (Light version of ADSWeb) (#17768)
* rough changes for adsweb-lite

* commit id of latest quality

* fix for aria label

* distro commit id changes

* merged latest changes distro id

* python path changes + quality name change

* Hide container views/settings/panel for ADSWeb

* undo unintentional change

* No python prompt changes for tsgops-image

* distro id changed for tsgops quality name

* changed quality name to tsgops

* changed name to tsgops-image

* python address change

* unintended changes undo

* correct python installation path for tsgops

* enable preview features tsgops

* no prompt for tsgops-lite image

* revert preview feature change

* Fixes for few comments

* removed extra line

* add deleted line

* Addressed comments

* Addressed final comments

* eslint error fix

* fix

Co-authored-by: Monica Gupta <mogupt@microsoft.com>
2022-02-02 16:57:59 -08:00
Charles Gagnon
4a2b31f3ba Hook up Notebook execution edits (#17943)
* Start rerouting VSCode cell execution APIs.

* Add more conversion code.

* Convert VSCode notebook registrations into ADS equivalents.

* Update vscode notebook provider kernels when notebook controller's supportedLanguages are set.

* Update an error message.

* Add another session argument.

* Add base classes for converting notebook serializers.

* Disable some vscode notebook methods.

* Disable more vscode APIs.

* Disable more stuff.

* Start implementing serializer notebook data conversions.

* Use direct references to extension host notebook methods, rather than azdata ones.

* Add a comment.

* Remove a space.

* Use import type to fix module loading errors.

* Use internal cancellation token class.

* Start adding cell output conversion.

* Convert data from byte array to a string.

* More output work.

* Use a Set for proxy filtering.

* Start adding tests.

* Include metadata in cell conversion. Fix other test failures.

* Fix serialize tests.

* Add more tests.

* Remove wildcard characters from vscode filenames.

* Start implementing session details.

* Add more kernel info.

* Add kernel spec.

* Add Future callback wrapper class.

* Start implementing execute conversion.

* Pass notebook URI to requestExecute.

* Start working on CellExecution methods.

* Move some code around to fix layering issues.

* Use proxy to access browser code, rather than direct imports.

* Move files around to fix layering issues.

* Remove unused imports.

* Start implementing some notebook cell execution behaviors.

* Revert some unnecessary extHost API changes.

* Check for nbformat.

* Also handle nbformat in serialize case.

* Active notebook extensions when resolving NotebookInput.

* Fix nbformat handling.

* Disable VSCode notebooks code.

* Filter out notebook services from registration assertion.

* Wait for providers to load before calling canResolve.

* Use controller's viewType for notebook provider ID, instead of controller ID.

* Start adding extHostNotebook tests for new APIs.

* Re-order proxy calls.

* Remove commented code.

* Move vscode provider files to browser folder. Fix RPC serialization issues by using readonly field instead of getter for providerId.

* Add a comment.

* Remove unnecessary dispose call.

* Handle disposable from registerExecuteProvider.

* Remove a comment.

* Remove unnecessary provider fields.

* Remove reference to notebook service to fix circular reference issue in stringify.

* Add object types for methods in ADSNotebookController.

* Wait for controller languages to be ready before marking session manager as ready.

* Add correct promise.

* Add undefined return type for optional supportedLanguages property.

* Refine promise logic.

* Move vscode functionality back to ExtHostNotebook, since the NotebookService can't be passed back over RPC (some kind of circular reference error).

* Fix remaining issues from last commit.

* Replace "not implemented" methods with placeholder return types in order to enable testing.

* Also wait for execution handler to be set before marking session manager as ready.

* Fix usage of NotebookRegistry when updating provider description languages.

* Refine file extension conversion.

* Fix file extension conversion to match ADS extension behavior.

* Emit new provider registration event when adding supported languages.

* Remove checks for duplicate file providers and kernels.

* Fix a test failure.

* Fix file extension parsing.

* Use default executeManager if one isn't defined for provider in notebookModel.

* Add descriptors for waiting on standardKernels registration.

* Increase timeout

* Add an error message.

* Start working on retrieving default kernel from registered providers, rather than always falling back to SQL.

* Revert "Start working on retrieving default kernel from registered providers, rather than always falling back to SQL."

This reverts commit 1916ea1ce3a0072f51bec683116dc7bb6c7aefdc.

* Emit activation events after provider registration.

* Wait on standard kernels availability when getting an execute provider.

* Throw an error if session manager isn't ready yet.

* Actually resolve language promise correctly.

* Add some checks for undefined notebook data objects.

* Create kernel spec data correctly.

* Add extension changes for local testing only.

* Clean up test class.

* Add a reminder comment.

* Undo commented out notebook stuff

* Temporarily hard code default kernel.

* Retrieve default kernel in notebookModel if it's not already provided.

* Revert an import change.

* Remove unnecessary method from extHostNotebook.

* Move an interface around.

* wip

* Check for proposed API for some VSCode extHost methods.

* Remove a comment.

* Fix notebookUtils tests.

* Fix notebookModel tests.

* Fix notebookFindModel tests.

* Fix notebookViewsExtension tests.

* Fix remaining notebookView tests.

* Refactor output conversion functionality into separate methods.

* Update some unit tests for output conversion.

* Move a method.

* Rename conversion methods to fit acronym styling.

* Add another conversion test case.

* Revert local testing changes.

* Remove old method.

* cleanup

* Remove some comments.

* Move localized string to locConstants.

* Add a space to loc string.

* Add comments to new SQL Carbon Edit tags.

* Create constants for nbformat and nbformat_minor.

* Move some vscode-only fields to proposed APIs.

* Check for valid state

* Properly null check

* Adding logging for provider wait timeouts.

* wip update

* Fix compile

* Switch to cell edits

* Update docs

* Remove custom output type

* cleanup

* fix

* cleanup

* more cleanup

* Fixes

* Fix tests and lint errors

Co-authored-by: Cory Rivera <corivera@microsoft.com>
2022-01-04 16:35:16 -08:00
Cory Rivera
2ecc3d35ca Initial implementation for VSCode Notebook support (#17885) 2022-01-03 15:59:37 -08:00
Aditya Bist
2a681605fa Updated credentials system (#17888)
* linting

* added flags

* remove testing values

* format

* format doc

* tested in linux

* remove unused interface

* comments

* review comments

* clean imports

* pr comments

* format doc

* changed promise location

* insiders december

* pr comments

* test ado change

* fix test

* comment out code for hygiene

* remove unused imports

* test creds from client only

* remove unused import

* trying enabling keytar

* trying enabling keytar

* disable in correct script

* print statements

* remove print statements

* check mock output

* add linux check

* remove print statements
2021-12-17 10:55:42 -08:00
Cory Rivera
5a4d1599db Improve Configure Python wizard experience for unfamiliar kernel names. (#17272) 2021-10-06 08:52:54 -07:00
Charles Gagnon
747b8e84a8 Add BDC spark connection logging (#17277)
* Add BDC spark connection logging

* fix test failures
2021-10-05 20:23:38 -07:00
Cory Rivera
14904bb671 Split up NotebookProvider into separate providers for handling file serialization and cell execution. (#17176) 2021-09-29 16:15:28 -07:00
Charles Gagnon
76e01fee60 Fix rest of notebook unhandled promises (#16933)
* Fix rest of notebook unhandled promises

* add rule

* fix some tests
2021-08-30 14:14:48 -07:00
Charles Gagnon
7804c87c87 Fix some unhandled promise lint errors (#16924) 2021-08-30 09:08:28 -07:00
Lucy Zhang
63d4cc0e80 don't show python upgrade prompt multiple times (#16080) 2021-07-15 06:50:46 -07:00
Lucy Zhang
6c5d35eaae Check kernel dependencies when a new session is started (#16040)
* check dependencies when a new session is started

* fix test

* fix issue when kernel spec is not found
2021-07-09 12:51:21 -07:00
Vasu Bhog
14ce88023e Add allow root setting for Ubuntu and Web Mode users (#16027)
* add allow root setting for Ubuntu and Web Mode users
2021-07-07 11:02:12 -07:00
Karl Burtram
31e9c21c06 Fix regressions in web build (#15920)
* Update build for web issues

* Fix remote\web package.json

* Remove extension activate test

* Add terminal entry point

* Bump distro

* Update docker files and pipelines

* Add linux task

* Update build pool

* Correct image name

* Update pipelines

* Fix indention

* Update sql-web-build.yml for Azure Pipelines

Add new build variable

* Update sql-web-build.yml for Azure Pipelines

Revert previous commit

* Disable sourcemaps upload

* Remove compile step

* Fix yaml typo

* Fix dependson error

* Update yaml

* Add notebook parameters

* Remove web from desktop builds

* Add web createDrop script

* Bump distro

* Change execution mode on createDrop script

* Code review feedback

* Update CI hygiene cache key

* Bump distro

* Fix merge conflicts

* Revert "Fix merge conflicts"

This reverts commit 06f7a58b6e0a065520b7686e8469e4e7682e157a.

* Bump distro to before smoke test update
2021-06-29 12:12:27 -07:00
Lucy Zhang
7ccbe61a1a Remove old python installation after new Python is successfully downloaded (#15931)
* remove old python after new download

* remove catch
2021-06-29 08:55:41 -07:00
Lucy Zhang
ff1a642157 Add jupyter server shutdown timeout setting (#15913)
* add jupyter server shutdown timeout setting

* no timeout if setting is 0

* set minimum value to 0
2021-06-25 15:16:32 -07:00
Lucy Zhang
0d2710341a increase jupyter server shutdown timeout (#15899) 2021-06-24 15:13:08 -07:00
Lucy Zhang
d6d2e7dc08 filter out prose-accelerator (#15633) 2021-06-08 11:48:57 -07:00
Lucy Zhang
43e8fde775 Update Python to 3.8.8 (#15278)
* update python fwlinks and remove bundle ver

* start fixing path for users with python 36

* prompt user for python version upgrade

* update python path after removing 3.6

* prompt users to upgrade and show pkg warning

* make prompt async

* remove python bundle ver from ML extension

* shutdown python 3.6 before deleting

* check useExistingPython before update prompt

* add dont ask again option

* remove 3.6 after installing 3.8

fix merge conflict

* give option to remove python36

* list user installed pip packages in warning

* create notebook to install pip packages

* update getPythonExePath method and add comments

* clean up code

* add comments

* pr comments

* add comment

* remove option to keep python36

* shutdown active servers before removing python36

* fix error removing old python w/ path change

* update to 3.8.10

* restart sessions for mac/linux
2021-05-19 15:15:07 -07:00
Lucy Zhang
f59e9b5695 Notebooks: Use new Python installation after configuration change (#14765)
* start new jupyter server

* restart session working (removed extra code)

* only restart server once

* shutdown session first then stop server

* add comments remove extra lines

* add comment

* fix test

* only restart jupyter sessions

* Dispose jupytersessionmanager and create new one

* move restart server logic out of notebookmodel

* move methods to azdata proposed

* pr comment
2021-03-24 15:31:20 -07:00
Cory Rivera
e2a5859155 Only include package versions in Manage Packages dialog if they're supported for the user's version of Python (#14584) 2021-03-08 18:05:10 -08:00
Lucy Zhang
8e07a6f239 pass in pythonexecutable when getting packages (#14550) 2021-03-04 15:26:46 -08:00
Lucy Zhang
74629c951d Notebooks: Fix pip installation not working (#14506)
* upgrade pip after python installation

* add comment

* only upgrade pip for new python installations

* reinstall pip version instead of upgrade to latest

* update comment

* get pip version through packages list
2021-03-03 17:44:45 -08:00
Lucy Zhang
800cd2fa89 catch getpythonuserdir error (#14193) 2021-02-10 10:13:08 -08:00
Chris LaFreniere
b29c07adb3 Always use \n when passing to Jupyter (#14185) 2021-02-08 09:38:06 -08:00
Charles Gagnon
49fa56369c Fixes for HDFS node expansion on BDC connections (#14174)
* Change cluster test connect to use endpoints route

* Add more logging

* More logging

* Only connect to controller if needed

* Add comments
2021-02-05 10:12:15 -08:00
Charles Gagnon
6907c8edab Use cryptography 3.2.1 for sparkmagic dependency (#14154)
* Use cryptography 3.2.1 for sparkmagic dependency

* pip -> conda

* fix typing

* fix tests

* prefix with required
2021-02-04 09:52:08 -08:00
Lucy Zhang
068649cba4 Change configure Jupyter server steps from async to sync (#13937)
* change config steps to sync

* fix tests

* use pathexistsSync

* remove pathExistsSync call

* address PR comments
2021-01-20 15:45:54 -08:00
Cory Rivera
9e9fac2991 Skip checking if python is running when only doing package upgrades during python install. (#13995) 2021-01-19 20:01:59 -08:00
Cory Rivera
94b697340d Remove unnecessary OutputChannel reference from PythonPathLookup class. (#13970) 2021-01-15 16:51:42 -08:00
Lucy Zhang
57446faa1e Notebooks: Improve Jupyter server start time (#13838)
* let jupyter server find port

* add undefined to return type

* remove time log

* add test for port number

* check that match[1] exists
2020-12-21 08:03:26 -08:00
Lucy Zhang
60925aa3a9 add . as trigger character (#13811) 2020-12-16 13:36:58 -08:00
Chris LaFreniere
a5231ec0e5 Notebook Extension: First logging improvements (#13729)
* First logging improvements

* PR feedback for err output
2020-12-11 11:21:06 -08:00
Chris LaFreniere
04117b2333 Remove prose from notebooks default package list (#13173) 2020-11-02 10:54:32 -08:00
Vasu Bhog
338beaff29 Update powershell kernel version to 0.1.4 (#13167)
* Update powershell kernel version to 0.1.4
2020-10-31 09:01:05 -07:00
Chris LaFreniere
dfb40e0159 Notebooks: Ensure Python Environment Variables are Removed Before First Python Command (#13097)
* Add CodeQL Analysis workflow (#10195)

* Add CodeQL Analysis workflow

* Fix path

* Ensure we delete python env vars correctly

* Move delete to top of method

Co-authored-by: Justin Hutchings <jhutchings1@users.noreply.github.com>
2020-10-30 11:06:46 -07:00
Vasu Bhog
cb30dd1893 Notebook Parameterization - Papermill Compatibility (#13034)
* Parameterization papermill fix

* Utilize isParameter instead

* Address PR comments, and fix tests

* Address comment
2020-10-23 20:32:55 -05:00
rajeshka
de7cc8ea53 Fix: Switching powershell notebooks reset kernel.json and failed to resolve kernel paths. (#13026)
* Set python path in kernel specs when running on SAW devices.

* Use tab spacer for kernel json.

* Update path to jupyter kernelspec.

* removing the kernelspec write

* Changed powershell kernel.json to use  appdata folder

* Addressed PR and added try catches around the code.

* removed redundant try catch

* removed redundant try catch

* removed another try catch

* removed space

* Fix for multiple powershell notebook failing issue

Co-authored-by: Cory Rivera <corivera@microsoft.com>
2020-10-21 17:36:47 -07:00
rajeshka
c4f649a849 Changes to use bundled python package (#12967)
* Set python path in kernel specs when running on SAW devices.

* Use tab spacer for kernel json.

* Update path to jupyter kernelspec.

* removing the kernelspec write

* Changed powershell kernel.json to use  appdata folder

* Addressed PR and added try catches around the code.

* removed redundant try catch

* removed redundant try catch

* removed another try catch

* removed space

Co-authored-by: Cory Rivera <corivera@microsoft.com>
2020-10-19 18:20:48 -07:00
Charles Gagnon
1c279675c8 Add support for low-privileged user to run spark notebooks (#12883)
* Add support for low-privileged user to run spark notebooks

* error

* fix test
2020-10-13 09:29:00 -07:00
Cory Rivera
108891ba2e Place current release at the top of version dropdown in Manage Packages dialog. (#12884)
* Also improved sorting for version numbers with non-numeric components.
2020-10-12 12:20:33 -07:00
Cory Rivera
9b7a1d7a26 Add more unit tests for JupyterServerInstallation (#12675) 2020-10-01 13:54:12 -07:00
Charles Gagnon
a91b965a33 Allow non-admin BDC connections to see BDC features (#12663)
* Add handling for non-admin BDC users

* Bump STS

* Fix HDFS root node commands

* remove nested awaits

* colon
2020-09-29 14:02:01 -07:00
Lucy Zhang
bfe694763b Add tests for Jupyter Session (#12053)
* add tests for Jupyter Session

* remove comment and fix description

* throw error if spark kernel provider is not mssql

* use localize strings and add pythonEnvVarPath test
2020-09-02 11:44:15 -07:00
Leila Lali
22c88cdd2e setting selected db name in drop down to current connection db name (#11912) 2020-08-31 14:44:08 -07:00
Lucy Zhang
a3121c0b2d Add tests for Jupyter Session (#11961)
* add tests for config kernel and connection

* move sinon.restore and fix config filepath
2020-08-25 18:59:53 -07:00
Vasu Bhog
29fbc7d952 Refactored Server Instance Tests (#11868)
* Refactored and improved testing through sinon

* Addressed changes - helper function and direct call to function refactor
2020-08-19 17:28:15 -05:00
Cory Rivera
0f063d3a2e Skip adding python user package directory to PATH on SAW devices. (#11849) 2020-08-18 17:14:19 -07:00