Commit Graph

47 Commits

Author SHA1 Message Date
Cory Rivera
e2327c393a Remove all Big Data Cluster features (#21369) 2022-12-07 12:28:17 -08:00
Aditya Bist
2a681605fa Updated credentials system (#17888)
* linting

* added flags

* remove testing values

* format

* format doc

* tested in linux

* remove unused interface

* comments

* review comments

* clean imports

* pr comments

* format doc

* changed promise location

* insiders december

* pr comments

* test ado change

* fix test

* comment out code for hygiene

* remove unused imports

* test creds from client only

* remove unused import

* trying enabling keytar

* trying enabling keytar

* disable in correct script

* print statements

* remove print statements

* check mock output

* add linux check

* remove print statements
2021-12-17 10:55:42 -08:00
Charles Gagnon
747b8e84a8 Add BDC spark connection logging (#17277)
* Add BDC spark connection logging

* fix test failures
2021-10-05 20:23:38 -07:00
Cory Rivera
14904bb671 Split up NotebookProvider into separate providers for handling file serialization and cell execution. (#17176) 2021-09-29 16:15:28 -07:00
Lucy Zhang
6c5d35eaae Check kernel dependencies when a new session is started (#16040)
* check dependencies when a new session is started

* fix test

* fix issue when kernel spec is not found
2021-07-09 12:51:21 -07:00
Lucy Zhang
f59e9b5695 Notebooks: Use new Python installation after configuration change (#14765)
* start new jupyter server

* restart session working (removed extra code)

* only restart server once

* shutdown session first then stop server

* add comments remove extra lines

* add comment

* fix test

* only restart jupyter sessions

* Dispose jupytersessionmanager and create new one

* move restart server logic out of notebookmodel

* move methods to azdata proposed

* pr comment
2021-03-24 15:31:20 -07:00
Chris LaFreniere
b29c07adb3 Always use \n when passing to Jupyter (#14185) 2021-02-08 09:38:06 -08:00
Charles Gagnon
49fa56369c Fixes for HDFS node expansion on BDC connections (#14174)
* Change cluster test connect to use endpoints route

* Add more logging

* More logging

* Only connect to controller if needed

* Add comments
2021-02-05 10:12:15 -08:00
Lucy Zhang
068649cba4 Change configure Jupyter server steps from async to sync (#13937)
* change config steps to sync

* fix tests

* use pathexistsSync

* remove pathExistsSync call

* address PR comments
2021-01-20 15:45:54 -08:00
Chris LaFreniere
a5231ec0e5 Notebook Extension: First logging improvements (#13729)
* First logging improvements

* PR feedback for err output
2020-12-11 11:21:06 -08:00
Charles Gagnon
1c279675c8 Add support for low-privileged user to run spark notebooks (#12883)
* Add support for low-privileged user to run spark notebooks

* error

* fix test
2020-10-13 09:29:00 -07:00
Charles Gagnon
a91b965a33 Allow non-admin BDC connections to see BDC features (#12663)
* Add handling for non-admin BDC users

* Bump STS

* Fix HDFS root node commands

* remove nested awaits

* colon
2020-09-29 14:02:01 -07:00
Lucy Zhang
bfe694763b Add tests for Jupyter Session (#12053)
* add tests for Jupyter Session

* remove comment and fix description

* throw error if spark kernel provider is not mssql

* use localize strings and add pythonEnvVarPath test
2020-09-02 11:44:15 -07:00
Lucy Zhang
a3121c0b2d Add tests for Jupyter Session (#11961)
* add tests for config kernel and connection

* move sinon.restore and fix config filepath
2020-08-25 18:59:53 -07:00
Cory Rivera
ff2d2d0339 Remove preview flag for Configure Python wizard. (#11813) 2020-08-14 12:36:40 -07:00
Chris LaFreniere
0c3c714be1 Stop hardcoding python3 (#11464) 2020-07-21 16:38:04 -07:00
Chris LaFreniere
960aa14027 stop running magics that dont exist (#10894) 2020-06-12 14:25:01 -07:00
Charles Gagnon
4d6ec69eba Fetch password from credential store before connecting to knox endpoint (#10666)
* Fetch password from credential store before connecting to knox endpoint

* Add combined code coverage script

* Revert "Add combined code coverage script"

This reverts commit e538f2fb2a79993a77f749f384fbc306b18b9f64.
2020-06-02 13:27:37 -07:00
Cory Rivera
98ce3c74e6 Hide installation options in Configure Python wizard if python is already setup. (#10635) 2020-05-29 17:28:02 -07:00
Charles Gagnon
f568ff82d8 Fix HDFS support for CU5+ BDC instances (#10577)
* Fix HDFS node auth for non-root username

* more changes
2020-05-27 10:17:28 -07:00
Cory Rivera
f94a9d0d58 Update Configure Python dialog to allow packages to be installed for specific kernels. (#10286) 2020-05-08 16:02:59 -07:00
Chris LaFreniere
0fbc3d1cb6 Get rid of red herrings (#10309) 2020-05-08 14:49:49 -07:00
Chris LaFreniere
0a3a0ec80f Notebooks: Support User-Installed Kernels (#10194)
* Fixes for attach to. Still need to update cache

* Don't have jupyter hardcoded

* Handle DOTNET_ROOT env var

* Fix on load

* Put behind feature flag

* Cleanup

* Error check

* PR feedback
2020-04-29 10:34:27 -07:00
Monica Gupta
63006e6e63 Undo default kqlmagic install (#9871)
Co-authored-by: Monica Gupta <mogupt@microsoft.com>
2020-04-07 10:20:37 -07:00
Monica Gupta
11c34f36d7 set ads context for kqlmagic (#9849)
Co-authored-by: Monica Gupta <mogupt@microsoft.com>
2020-04-06 09:48:16 -07:00
Monica Gupta
9c842e7548 Kql magic updates (#9753)
* Inludes package upgrade and silent env

* Make user param changes

* Address comments

Co-authored-by: Monica Gupta <mogupt@microsoft.com>
2020-03-30 10:43:26 -07:00
Monica Gupta
fa6898ded7 Pre-installs and registers kqlmagic (#9558)
Co-authored-by: Monica Gupta <mogupt@microsoft.com>
2020-03-17 09:02:58 -07:00
Charles Gagnon
b1d58767c4 Add livy_session_startup_timeout_seconds config back in (#9212) 2020-02-25 11:31:10 -08:00
Charles Gagnon
abbb1e54da Add config for ignoring SSL errors on BDC queries (#8169)
* Add config for ignoring SSL errors on BDC queries

* Fix error handling in write stream

* Disable tslint check

* Handle promise appropriately

* PR comments

* Change defaults to true
2019-11-01 15:20:47 -07:00
Kevin Cunnane
2558d6bff6 Fix #8162 Spark timeout in BDC cluster is higher than client-side, causing successful sessions to show as failed (#8164)
* Fix #8162 Spark timeout in BDC cluster is higher than client-side, causing successful sessions to show as failed
2019-10-31 17:00:36 -07:00
Chris LaFreniere
5aff7ef4c7 Ensure cwd matches notebook path (#8137)
* Ensure cwd matches notebook path

* add error checking

* silent, don't store history
2019-10-31 15:10:49 -07:00
Chris LaFreniere
fb4fccf2d5 Notebooks: ensure python path dirs added to path on session start (#7968)
* ensure python path dirs add to path session start

* Change logic slightly

* PR feedback from Charles
2019-10-23 19:22:24 -07:00
Charles Gagnon
4a68ab4659 Hygiene linting for extensions + new rule (#7843)
* linting for extensions + new rule

* Remove unneeded array

* Fix spelling mistake

* Fix bad merge
2019-10-22 18:56:31 -07:00
Chris LaFreniere
e3ae5263c6 Starting Fewer Jupyter Servers for Notebooks (#7744)
* Start fewer Jupyter servers

* Windows fix for drive casing

* PR Feedback

* Quick fix

* Fixing bug

* Ensure environment variables set 4 session startup

* test fix

* Dummy commit to update comment
2019-10-20 21:38:58 -07:00
Charles Gagnon
f418104b7a big data cluster -> Big Data Cluster (#7536) 2019-10-07 13:13:40 -07:00
Cory Rivera
79d0239362 Use python3 as the default python version in Spark sessions. (#7353) 2019-09-25 11:23:39 -07:00
Kevin Cunnane
1f00249646 Fix HDFS with AD auth for browse, read file scenarios (#6840)
* Fix HDFS with AD auth for browse, read
- HDFS now fully supports expanding nodes for all levels, including using cookie for auth
- HDFS now support reading files from HDFS
- HDFS write file is broken and will be fixed (either in PR update or separate PR)
- Removed hack to use gateway-0 instead of actual DNS name now these are supported. Needed for testing

* Fix Jupyter error using new DMV with endpoints
2019-08-20 18:12:38 -07:00
Kevin Cunnane
fdac1d5382 Re-add AD auth for SQL big data cluster (#6759)
* Revert "Revert "Initial AD support for BDCs (#6741)" as it breaks linux (#6758)"

This reverts commit 51660b25ef.

* Install kerberos package on linux
2019-08-14 20:19:31 -07:00
Kevin Cunnane
51660b25ef Revert "Initial AD support for BDCs (#6741)" as it breaks linux (#6758)
This reverts commit 52f8984a99.
2019-08-14 19:16:52 -07:00
Kevin Cunnane
52f8984a99 Initial AD support for BDCs (#6741)
Partially working AD support for BDCs with some known issues
- Plumbed through kerberos support to Notebooks.
  - Using "gateway-0" for service temporarily as service endpoints API doesn't yet return correct DNS name. Will update in separate PR once available
- Plumbed kerberos auth to HDFS, Spark. Only partially working as we use same token on each call 
  - Will fix in separate PR, as this requires a refactor of WebHDFS library. Will need to either get new token every time or set a cookie, both of which require refactors
- Fixed error when Data Service node expansion failed and blocked all OE expansion
- Support for SqlToolsService change to use new cluster endpoints DMV
  -  Updated API to add new endpoints field to replace IP + port
  - Added logic to handle case where endpoints for Yarn, Grafana etc. are in the list
  - Sort list and use expected new localized strings

- Updated SqlToolsService to include support for new DMV
- Add "gateway-0" handling in Jupyter session as workaround for lack of domain names in endpoints list
2019-08-14 18:09:41 -07:00
Kevin Cunnane
b3e8f466ec Replace Big Data Cluster with big data cluster (#6467)
* Replace Big Data Cluster with big data cluster
Official docs guidance is to use "big data cluster" instead of "Big Data Cluster"

* Use doublequotes and full product name
2019-07-23 11:54:33 -07:00
Anthony Dresser
58959ef35e Enable stricter compile options on extensions (#5044)
* enable stricter compile settings in extensions

* more strict compile

* formatting

* formatting

* revert some changes

* formtting

* formatting
2019-04-23 11:18:00 -07:00
Gene Lee
b3be1d79cd Add support for new endpoint key string 'gateway' (#4954) 2019-04-09 15:19:02 -07:00
Chris LaFreniere
1e989060f9 Rewrite Spark UI link when using unified connection (#4362)
* Rewrite Spark UI link when using unified connection

* Add more robust error checking
2019-03-08 17:34:47 -08:00
Yurong He
a48e9bc64c Fixed #4206 and #4207 open and close notebook quickly issue #4240 (#4255)
* Fixed  #4206 and #4207

* Check if it makes my PR run tests

* Add isReady to JupyterSessionManager
2019-03-04 12:46:27 -08:00
Karl Burtram
84890eb1b4 Update product references from 'sqlops' to 'azdata' (#4259)
* Update extensions to use azdata

* Switch core code to use azdata
2019-03-01 13:59:37 -08:00
Cory Rivera
70838c3e24 Move SQL 2019 extension's notebook code into Azure Data Studio (#4090) 2019-02-20 10:55:49 -08:00