Commit Graph

62 Commits

Author SHA1 Message Date
Chris LaFreniere
738c8231ee Always send \n instead of \r\n to Jupyter kernel (#7995)
* Always send \n instead of \r\n to kernel

* Use replace instead of split/join
2019-10-24 22:43:46 -07:00
Chris LaFreniere
c9dd2fb4a8 Notebooks: ensure python path dirs added to path on session start (#7968)
* ensure python path dirs add to path session start

* Change logic slightly

* PR feedback from Charles
2019-10-24 00:11:38 -07:00
Chris LaFreniere
3d95b0032e Update recommended package versions for ps (#7972) 2019-10-24 00:11:31 -07:00
Cory Rivera
10d7db8b37 Append stdErr log to error message when a streamed console command fails. (#7868) 2019-10-24 00:06:33 -07:00
Chris LaFreniere
e3ae5263c6 Starting Fewer Jupyter Servers for Notebooks (#7744)
* Start fewer Jupyter servers

* Windows fix for drive casing

* PR Feedback

* Quick fix

* Fixing bug

* Ensure environment variables set 4 session startup

* test fix

* Dummy commit to update comment
2019-10-20 21:38:58 -07:00
Cory Rivera
9bdaba3b65 Remove sqlmlutils from default packages due to installation failures on OSX. (#7780) 2019-10-17 17:21:53 -07:00
Cory Rivera
0b039830ea Add sqlmlutils to list of required notebook packages. (#7740) 2019-10-16 14:16:09 -07:00
Cory Rivera
23861bd369 Wait for python installs to complete before starting a python notebook. (#7729) 2019-10-15 17:06:19 -07:00
Cory Rivera
b6ef5469de Correctly handle exceptions in package upgrade promise. (#7734) 2019-10-15 13:50:30 -07:00
Cory Rivera
eb465fde1a Skip prompting for package upgrade if a python install is already in progress. (#7717) 2019-10-14 15:31:53 -07:00
Cory Rivera
9a3f72591e Show a background task when upgrading python packages. (#7649) 2019-10-11 10:56:19 -07:00
Cory Rivera
3a01f960a7 Add prompt for user to upgrade python packages when starting a notebook (#7574) 2019-10-08 17:42:10 -07:00
Charles Gagnon
f418104b7a big data cluster -> Big Data Cluster (#7536) 2019-10-07 13:13:40 -07:00
Cory Rivera
826c4115a7 Add powershell package to notebook python setup. (#7503)
Also added force-reinstall flag to installs when running Reinstall Dependencies.
2019-10-04 11:08:07 -07:00
Cory Rivera
79d0239362 Use python3 as the default python version in Spark sessions. (#7353) 2019-09-25 11:23:39 -07:00
Cory Rivera
094d6f2339 Update python packages to use sparkmagic 0.12.9 (#7240) 2019-09-17 15:05:59 -07:00
Anthony Dresser
28d453fced Implement a no sync rule (#7216)
* implement a no sync rule

* fix linting disable

* fix unused imports

* exclude more testing

* clean up fs usage

* clean up more fs usage

* remove duplicate of code

* fix compile errors
2019-09-17 13:32:42 -07:00
Chris LaFreniere
2d6f69da1c no mathjax (#6995) 2019-08-29 18:07:05 -07:00
Kevin Cunnane
1f00249646 Fix HDFS with AD auth for browse, read file scenarios (#6840)
* Fix HDFS with AD auth for browse, read
- HDFS now fully supports expanding nodes for all levels, including using cookie for auth
- HDFS now support reading files from HDFS
- HDFS write file is broken and will be fixed (either in PR update or separate PR)
- Removed hack to use gateway-0 instead of actual DNS name now these are supported. Needed for testing

* Fix Jupyter error using new DMV with endpoints
2019-08-20 18:12:38 -07:00
Kevin Cunnane
fdac1d5382 Re-add AD auth for SQL big data cluster (#6759)
* Revert "Revert "Initial AD support for BDCs (#6741)" as it breaks linux (#6758)"

This reverts commit 51660b25ef.

* Install kerberos package on linux
2019-08-14 20:19:31 -07:00
Kevin Cunnane
51660b25ef Revert "Initial AD support for BDCs (#6741)" as it breaks linux (#6758)
This reverts commit 52f8984a99.
2019-08-14 19:16:52 -07:00
Kevin Cunnane
52f8984a99 Initial AD support for BDCs (#6741)
Partially working AD support for BDCs with some known issues
- Plumbed through kerberos support to Notebooks.
  - Using "gateway-0" for service temporarily as service endpoints API doesn't yet return correct DNS name. Will update in separate PR once available
- Plumbed kerberos auth to HDFS, Spark. Only partially working as we use same token on each call 
  - Will fix in separate PR, as this requires a refactor of WebHDFS library. Will need to either get new token every time or set a cookie, both of which require refactors
- Fixed error when Data Service node expansion failed and blocked all OE expansion
- Support for SqlToolsService change to use new cluster endpoints DMV
  -  Updated API to add new endpoints field to replace IP + port
  - Added logic to handle case where endpoints for Yarn, Grafana etc. are in the list
  - Sort list and use expected new localized strings

- Updated SqlToolsService to include support for new DMV
- Add "gateway-0" handling in Jupyter session as workaround for lack of domain names in endpoints list
2019-08-14 18:09:41 -07:00
Cory Rivera
1ff9a2ec4c Clarify some error messages in deployment notebooks and python installation. (#6658) 2019-08-08 16:54:14 -07:00
Cory Rivera
08b678b522 Reinstall pip during python install on Windows to update shebangs in exe files. (#6494) 2019-07-25 11:07:22 -07:00
Kevin Cunnane
b3e8f466ec Replace Big Data Cluster with big data cluster (#6467)
* Replace Big Data Cluster with big data cluster
Official docs guidance is to use "big data cluster" instead of "Big Data Cluster"

* Use doublequotes and full product name
2019-07-23 11:54:33 -07:00
Cory Rivera
b313cb58db Check if python is running using powershell commands before doing install on Windows. (#6405) 2019-07-19 14:28:34 -07:00
Alan Ren
0c5d5f1dd5 pass environment variables to Notebooks from ADS (#6389)
* get the new environment variables for notebook

* comments

* Update serverInstance.ts

address comments

* be more specific when taking env variables
2019-07-18 10:40:07 -07:00
Cory Rivera
2d0b805549 Check if python executable exists before querying user package directory. (#6345) 2019-07-09 17:58:26 -07:00
Cory Rivera
5bf62978eb Fix python install issues caused by other preexisting Python versions. (#6294)
* Remove --user option when doing pip installs for our standalone Python version.

* Use force-reinstall option when installing sparkmagic since we use a custom version.

* Use force-reinstall when installing pip packages from Manage Packages dialog so that dependencies don't get split across multiple locations.

* Update PATH after install to include additional package directories.
2019-07-09 17:05:00 -07:00
Cory Rivera
4189e761ff Fix bugs in selecting a system version of Python for Notebook dependencies (#6250) 2019-07-03 12:51:48 -07:00
Cory Rivera
72c3239d63 Allow user to select source package type in Manage Packages dialog. (#6092) 2019-06-20 10:51:36 -07:00
Cory Rivera
cecc899949 Disable Manage Packages button if python is not installed (#6008) 2019-06-17 18:28:16 -07:00
Cory Rivera
ff38bc8143 Add dialog to notebooks for managing Pip packages (#5944) 2019-06-10 17:02:31 -07:00
Cory Rivera
fb713e0762 Automatically detect existing Python/Conda installs in Configure Python dialog. (#5801) 2019-06-03 11:56:06 -07:00
Cory Rivera
a59d1d3c05 Add functionality to use an existing Python installation for Notebook dependencies (#5228) 2019-05-17 14:39:44 -07:00
Kevin Cunnane
64416e05c1 Notebook StdIn support to fix #5231 (#5232)
Fixes #5231 
- Add stdin handling. Has to be at UI level so add plumb through handling
- Add unit tests
- Add new StdIn component.

Testing:
Unit Tests and manual testing of following:
- Prompt for password using `getpass` in python.
   - Password prompt is hidden since "password" is true.
   - Hit enter, it completes
- prompt, stop cell running, StdIn disappears
- prompt, hit escape, stdIn disappears and stdIn request is handled.

Issues: focus isn't always set to the input even though we call focus.
Will investigate this further.
2019-04-30 14:57:27 -07:00
Chris LaFreniere
e72d0d03ed Notebooks: More Jupyter Server Hardening (#5264)
* Do not rely on same starting port every time, misc

* put back to 5 seconds for process kill timeout

* extHostNotebook shutdown manager handle
2019-04-30 14:13:04 -07:00
Chris LaFreniere
bb9c85cd8f Improve Cleanup of Jupyter processes on Notebook and/or ADS Close (#5142)
* Close jupyter and python

* Ensure we stop jupyter correctly on process end

* dont stopServer from clientSession shutdown

* PR comments

* close notebook after each test
2019-04-26 15:28:26 -07:00
Anthony Dresser
58959ef35e Enable stricter compile options on extensions (#5044)
* enable stricter compile settings in extensions

* more strict compile

* formatting

* formatting

* revert some changes

* formtting

* formatting
2019-04-23 11:18:00 -07:00
Cory Rivera
1b24dff738 Show an Install Skipped message when Python already exists at the specified install location. (#5141)
* Also fixed a bug where future installs would get blocked after doing an install that gets skipped.
2019-04-22 14:28:59 -07:00
Cory Rivera
880e3e10da Update Python install messages to differentiate between download completion and install completion. (#5114) 2019-04-18 16:52:50 -07:00
Anthony Dresser
ddd89fc52a Renable Strict TSLint (#5018)
* removes more builder references

* remove builder from profiler

* formatting

* fix profiler dailog

* remove builder from oatuhdialog

* remove the rest of builder references

* formatting

* add more strict null checks to base

* enable strict tslint rules

* fix formatting

* fix compile error

* fix the rest of the hygeny issues and add pipeline step

* fix pipeline files
2019-04-18 00:34:53 -07:00
Cory Rivera
1870d83081 Add additional error handling to Python installation for Notebooks (#4891)
* Also enabled integration tests for python installation.
2019-04-10 17:09:28 -07:00
Kevin Cunnane
37f45b10a3 Mitigate (but not fully fix) Run Cell from disconnected notebook (#4960)
This is a partial fix that lays groundwork for full "Prompt to connect" if a kernel needs a connection.
I am waiting on Yurong's refactoring of connection handling before doing any of the prompt work.

- Adds kernel metadata about whether a connection is required.
- For Jupyter, only Spark kernels are listed as requiring a connection
- If this is true and there's no active connection, will show notification and not call execute

In the future, this path will still be used if user is prompted to connect and cancels out.
The future change will be to inject a "connect" handler from notebook.component to the cell callback and use to set connection context
2019-04-09 17:45:05 -07:00
Gene Lee
b3be1d79cd Add support for new endpoint key string 'gateway' (#4954) 2019-04-09 15:19:02 -07:00
Cory Rivera
c7bc37d010 Add quotes around python paths to guard against spaces. (#4775) 2019-03-29 14:55:08 -07:00
Cory Rivera
37ab493b78 Use getErrorMessage to get exception messages in python installation code. (#4730) 2019-03-27 12:14:42 -07:00
Anthony Dresser
4472764f3a extensions tslint cleanup/Connection config refactor (#4370)
* various clean ups

* formatting

* remove linting

* formatting

* IConfigurationService is even better

* messing with connection config tests

* update tests

* formatting

* foramtting

* remove unused code

* add more tests
2019-03-20 11:59:07 -07:00
Kevin Cunnane
6f1a03587a Fix #4029 Ensure changeKernels always resolves, even in error states (#4488)
* Fix #4029 Ensure changeKernels always resolves, even in error states
This is necessary to unblock reverting the kernel on canceling  Python install
- startSession now correctly sets up kernel information, since a kernel is loaded there.
- Remove call to change kernel on session initialize. This isn't needed due to refactor
- Handle kernel change failure by attempting to fall back to old kernel
- ExtensionHost $startNewSession now ensures errors are sent across the wire.
- Update AttachTo and Kernel dropdowns so they handle kernel being available. This is needed since other changes mean the session is likely ready before these get going

* Fix to handle python cancel and load existing scenarios
- Made changes to handle failure flow when Python dialog is canceled
- Made changes to handle initial load fail. Kernel and Attach To dropdowns show No Kernel / None and you can choose a kernel
- Added error wrapping in ext host so that string errors make it across and aren't lost.
2019-03-14 13:07:08 -07:00
Cory Rivera
34d36c1de1 Prompt for Python installation after choosing a Jupyter kernel in notebook (#4453) 2019-03-13 18:44:54 -07:00