diff --git a/extensions/mssql/notebooks/TSG/cluster-status.ipynb b/extensions/mssql/notebooks/TSG/cluster-status.ipynb index ed2a6da2ac..cf5d0d97ed 100644 --- a/extensions/mssql/notebooks/TSG/cluster-status.ipynb +++ b/extensions/mssql/notebooks/TSG/cluster-status.ipynb @@ -22,22 +22,17 @@ "cells": [ { "cell_type": "markdown", - "source": "\n\n# View the status of your big data cluster\nThis notebook allows you to see the status of the controller, master instance, and pools in your SQL Server big data cluster.\n\n## Important Instructions\n### **Before you begin, you will need:**\n* Big data cluster name\n* Controller username\n* Controller password\n* Controller endpoint \n\nYou can find the controller endpoint from the big data cluster dashboard in the Service Endpoints table. The endpoint is listed as **Cluster Management Service.**\n\nIf you do not know the credentials, ask the admin who deployed your cluster.\n\n### **Instructions**\n* For the best experience, click **Run Cells** on the toolbar above. This will automatically execute all code cells below and show the cluster status in each table.\n* When you click **Run Cells** for this Notebook, you will be prompted at the *Log in to your big data cluster* code cell to provide your login credentials. Follow the prompts and press enter to proceed.\n* **You won't need to modify any of the code cell contents** in this Notebook. If you accidentally made a change, you can reopen this Notebook from the bdc dashboard.\n\n\n", + "source": "\n\n# View the status of your big data cluster\nThis notebook allows you to see the status of the controller, master instance, and pools in your SQL Server big data cluster.\n\n## Important Instructions\n### **Before you begin, you will need:**\n* Big data cluster name\n* Controller username\n* Controller password\n* Controller endpoint \n\nYou can find the controller endpoint from the big data cluster dashboard in the Service Endpoints table. The endpoint is listed as **Cluster Management Service.**\n\nIf you do not know the credentials, ask the admin who deployed your cluster.\n\n### **Prerequisites**\nEnsure the following tools are installed and added to PATH before proceeding.\n\n|Tools|Description|Installation|\n|---|---|---|\n|kubectl | Command-line tool for monitoring the underlying Kuberentes cluster | [Installation](https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl-binary-using-native-package-management) |\n|azdata | Command-line tool for installing and managing a big data cluster |[Installation](https://docs.microsoft.com/en-us/sql/big-data-cluster/deploy-install-azdata?view=sqlallproducts-allversions) |\n\n### **Instructions**\n* For the best experience, click **Run Cells** on the toolbar above. This will automatically execute all code cells below and show the cluster status in each table.\n* When you click **Run Cells** for this Notebook, you will be prompted at the *Log in to your big data cluster* code cell to provide your login credentials. Follow the prompts and press enter to proceed.\n* **You won't need to modify any of the code cell contents** in this Notebook. If you accidentally made a change, you can reopen this Notebook from the bdc dashboard.\n\n\n", "metadata": {} }, { "cell_type": "markdown", - "source": "## **Dependencies**\r\n\r\n> This Notebook will try to install these dependencies for you.\r\n\r\n----------------------------------------------------------------\r\n
| Tool | \r\nRequired | \r\nDescription | \r\n
|---|---|---|
| mssqlctl | \r\nYes | \r\nCommand-line tool for installing and managing a big data cluster. | \r\n
| pandas | \r\nYes | \r\nPython Library for formatting data (More info). | \r\n
",
- "metadata": {}
- },
- {
- "cell_type": "markdown",
- "source": "### **Install latest version of mssqlctl**",
+ "source": "### **azdata version**",
"metadata": {}
},
{
"cell_type": "code",
- "source": "import sys, platform\r\n\r\nif platform.system()==\"Windows\":\r\n user = ' --user'\r\nelse:\r\n user = ''\r\n\r\ndef executeCommand(cmd, successMsgs, printMsg):\r\n print(printMsg)\r\n cmdOutput = !{cmd}\r\n cmdOutput = ''.join(cmdOutput)\r\n if any(msg in cmdOutput for msg in successMsgs):\r\n print(f\"\\nSuccess >> \" + cmd)\r\n else:\r\n raise SystemExit(f'\\nFailed during:\\n\\n\\t{cmd}\\n\\nreturned: \\n' + ''.join(cmdOutput) + '.\\n')\r\n\r\ninstallPath = 'https://private-repo.microsoft.com/python/ctp3.1/mssqlctl/requirements.txt'\r\ncmd = f'{sys.executable} -m pip uninstall --yes mssqlctl-cli-storage'\r\ncmdOutput = !{cmd}\r\n\r\ncmd = f'{sys.executable} -m pip uninstall -r {installPath} --yes'\r\nexecuteCommand(cmd, ['is not installed', 'Successfully uninstalled mssqlctl'], 'Uninstalling mssqlctl:')\r\n\r\ncmd = f'{sys.executable} -m pip install -r {installPath}{user} --trusted-host helsinki'\r\ncmdOutput = !{cmd}\r\nexecuteCommand(cmd, ['Requirement already satisfied', 'Successfully installed mssqlctl'], 'Installing the latest version of mssqlctl:')",
+ "source": "import sys, os\r\ncmd = f'azdata --version'\r\ncmdOutput = !{cmd}\r\nazdataStr = '\\'azdata\\''\r\nif len(cmdOutput) > 0 and ('command not found' in cmdOutput[1] or f'{azdataStr} is not recognized as an internal or external command' in cmdOutput[0]):\r\n raise SystemExit('azdata not found! Please make sure azdata is installed and added to path' + '.\\n')\r\nif '15.0' in cmdOutput[0]:\r\n print('azdata version: ' + cmdOutput[0])\r\n",
"metadata": {},
"outputs": [],
"execution_count": 0
@@ -61,7 +56,7 @@
},
{
"cell_type": "code",
- "source": "import os, getpass, json\nimport pandas as pd\nimport numpy as np\nfrom IPython.display import *\n\ndef PromptForInfo(promptMsg, isPassword, errorMsg):\n if isPassword:\n promptResponse = getpass.getpass(prompt=promptMsg)\n else:\n promptResponse = input(promptMsg)\n if promptResponse == \"\":\n raise SystemExit(errorMsg + '\\n')\n return promptResponse\n\n# Prompt user inputs:\ncluster_name = PromptForInfo('Please provide your Cluster Name: ', False, 'Cluster Name is required!')\n\ncontroller_username = PromptForInfo('Please provide your Controller Username for login: ', False, 'Controller Username is required!')\n\ncontroller_password = PromptForInfo('Controller Password: ', True, 'Password is required!')\nprint('***********')\n\n!mssqlctl logout\n# Login in to your big data cluster \ncmd = f'mssqlctl login -n {cluster_name} -u {controller_username} -a yes'\nprint(\"Start \" + cmd)\nos.environ['CONTROLLER_USERNAME'] = controller_username\nos.environ['CONTROLLER_PASSWORD'] = controller_password\nos.environ['ACCEPT_EULA'] = 'yes'\n\nloginResult = !{cmd}\nif 'ERROR: Please check your kube config or specify the correct controller endpoint with: --controller-endpoint https:// ",
+ "source": "## **Cluster Status**\r\nFor each cluster component below, running each code cell will generate a table. This table will include:\r\n\r\n|Column Name|Description|\r\n|---|---|\r\n|**Kind** | Identifies if component is a pod or a set. |\r\n|**LogsURL** | Link to [Kibana](https://www.elastic.co/guide/en/kibana/current/introduction.html) logs which is used for troubleshooting. |\r\n|**Name** | Provides the specific name of the pod or set. |\r\n|**NodeMetricsURL** | Link to [Grafana](https://grafana.com/docs/guides/basic_concepts/) dashboard to view key metrics of the node. |\r\n|**SQLMetricsURL** | Link to [Grafana](https://grafana.com/docs/guides/basic_concepts/) dashboard to view key metrics of the SQL instance. |\r\n|**State** | Indicates state of the pod or set. |\r\n\r\n----------------------------------------------------------------",
"metadata": {}
},
{
@@ -90,7 +85,7 @@
},
{
"cell_type": "code",
- "source": "# Display status of controller\nresults = !mssqlctl bdc control status show\nshow_results(results)",
+ "source": "# Display status of controller\nresults = !azdata bdc control status show\nshow_results(results)",
"metadata": {},
"outputs": [],
"execution_count": 0
@@ -102,7 +97,7 @@
},
{
"cell_type": "code",
- "source": "# Display status of master instance\nresults = !mssqlctl bdc pool status show -k master -n default\nshow_results(results)",
+ "source": "# Display status of master instance\nresults = !azdata bdc pool status show -k master -n default\nshow_results(results)",
"metadata": {},
"outputs": [],
"execution_count": 0
@@ -114,7 +109,7 @@
},
{
"cell_type": "code",
- "source": "# Display status of compute pool\nresults = !mssqlctl bdc pool status show -k compute -n default\nshow_results(results)",
+ "source": "# Display status of compute pool\nresults = !azdata bdc pool status show -k compute -n default\nshow_results(results)",
"metadata": {},
"outputs": [],
"execution_count": 0
@@ -126,7 +121,7 @@
},
{
"cell_type": "code",
- "source": "# Display status of storage pools\nresults = !mssqlctl bdc pool status show -k storage -n default\nshow_results(results)",
+ "source": "# Display status of storage pools\nresults = !azdata bdc pool status show -k storage -n default\nshow_results(results)",
"metadata": {},
"outputs": [],
"execution_count": 0
@@ -138,7 +133,7 @@
},
{
"cell_type": "code",
- "source": "# Display status of data pools\nresults = !mssqlctl bdc pool status show -k data -n default\nshow_results(results)",
+ "source": "# Display status of data pools\nresults = !azdata bdc pool status show -k data -n default\nshow_results(results)",
"metadata": {},
"outputs": [],
"execution_count": 0
@@ -150,7 +145,7 @@
},
{
"cell_type": "code",
- "source": "# Display status of spark pool\nif spark_exists:\n results = !mssqlctl bdc pool status show -k spark -n default\n show_results(results)\nelse:\n print('No spark pool.')",
+ "source": "# Display status of spark pool\nif spark_exists:\n results = !azdata bdc pool status show -k spark -n default\n show_results(results)\nelse:\n print('No spark pool.')",
"metadata": {},
"outputs": [],
"execution_count": 0
\r\n
\r\n\r\n \r\n\r\n\r\nColumn Name \r\nDescription \r\n\r\n \r\nKind \r\nIdentifies if component is a pod or a set. \r\n\r\n \r\nLogsURL \r\nLink to Kibana logs which is used for troubleshooting. \r\n\r\n \r\nName \r\nProvides the specific name of the pod or set. \r\n\r\n \r\nNodeMetricsURL \r\nLink to Grafana dashboard to view key metrics of the node. \r\n\r\n \r\nSQLMetricsURL \r\nLink to Grafana dashboard to view key metrics of the SQL instance \r\n\r\n \r\n\r\nState \r\nIndicates state of the pod or set.. \r\n