azuredatastudio/extensions/resource-deployment/notebooks/bdc/2019/CTP3-1/deploy-bdc-existing-cluster.ipynb

{
    "metadata": {
        "kernelspec": {
            "name": "python3",
            "display_name": "Python 3"
        },
        "language_info": {
            "name": "python",
            "version": "3.6.6",
            "mimetype": "text/x-python",
            "codemirror_mode": {
                "name": "ipython",
                "version": 3
            },
            "pygments_lexer": "ipython3",
            "nbconvert_exporter": "python",
            "file_extension": ".py"
        }
    },
    "nbformat_minor": 2,
    "nbformat": 4,
    "cells": [
        {
            "cell_type": "markdown",
            "source": "![Microsoft](https://raw.githubusercontent.com/microsoft/azuredatastudio/master/src/sql/media/microsoft-small-logo.png)\n\n## Deploy SQL Server 2019 big data cluster on an existing Kubernetes cluster\n\nThis notebook walks through the process of deploying a SQL Server 2019 big data cluster on an existing Kubernetes cluster\n\n* Follow the instructions in the **Dependencies** cell to install the dependencies.\n* Make sure you have the target cluster set as the current context in your kubectl config file.\n* The **Required information** cell will prompt you for the required information to create a SQL Server 2019 big data cluster.\n\n",
            "metadata": {}
        },
        {
            "cell_type": "markdown",
            "source": "### **Dependencies**\n<table>\n<colgroup>\n<col style=\"width: 10%\" />\n<col style=\"width: 85%\" />\n<col style=\"width: 10%\" />\n</colgroup>\n<thead>\n<tr class=\"header\">\n<th>Tool</th>\n<th>Description</th>\n<th>Installation</th>\n</tr>\n</thead>\n<tbody>\n<td><strong>kubectl</strong></td>\n<td>Command-line tool for monitoring the underlying Kuberentes cluster (<a href=\"https://kubernetes.io/docs/tasks/tools/install-kubectl/\">More info</a>)</td>\n<td><a href=\"https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl-binary-using-native-package-management\">Install</a></td>\n</tr>\n<tr>\n<td><strong>mssqlctl</strong></td>\n<td>Command-line tool for installing and managing a big data cluster</td>\n<td><a href=\"https://docs.microsoft.com/en-us/sql/big-data-cluster/deploy-install-mssqlctl?view=sqlallproducts-allversions\">Install</a></td>\n</tr>\n</tbody>\n</table>\n<p>",
            "metadata": {}
        },
        {
            "cell_type": "markdown",
            "source": "### **Check dependencies**",
            "metadata": {}
        },
        {
            "cell_type": "code",
            "source": "#Run command helper function\r\ndef run_command():\r\n    print(\"Executing: \" + cmd)\r\n    !{cmd}\r\n    if _exit_code != 0:\r\n        raise SystemExit(f'Shell command:\\n\\n\\t{cmd}\\n\\nreturned non-zero exit code: ' + str(_exit_code) + '.\\n')\r\n    print(f'Successfully executed: {cmd}')\r\n\r\ncmd = 'kubectl version --client=true'\r\nrun_command()\r\ncmd = 'mssqlctl --version'\r\nrun_command()",
            "metadata": {},
            "outputs": [],
            "execution_count": 1
        },
        {
            "cell_type": "markdown",
            "source": "### **Show current context**",
            "metadata": {}
        },
        {
            "cell_type": "code",
            "source": "cmd = ' kubectl config current-context'\r\nrun_command()",
            "metadata": {},
            "outputs": [],
            "execution_count": 2
        },
        {
            "cell_type": "markdown",
            "source": "### **Required information**",
            "metadata": {}
        },
        {
            "cell_type": "code",
            "source": "import getpass\ndef get_user_input(input_name, is_password = False, confirm_password = False):\n    if is_password:\n        user_input = getpass.getpass(prompt = input_name)\n        if confirm_password:\n            user_input_confirm = getpass.getpass(prompt = 'Confirm '+ input_name)\n            if user_input != user_input_confirm:\n                raise SystemExit(f'{input_name} does not match the confirmation password')\n        print(f'{input_name}: ******')\n    else:\n        user_input = input(input_name)\n        print(input_name + ': ' + user_input)\n    if user_input == \"\":\n        raise SystemExit(f'{input_name} is required')\n    return user_input;\n\nmssql_cluster_name = get_user_input('Cluster name')\nbdc_controller_username = get_user_input('Controller username')\nbdc_password = get_user_input('Controller password', True, True)\nprint('Knox and SQL Server will use the same password')\ndocker_username = get_user_input('Docker username')\ndocker_password = get_user_input('Docker password', True)",
            "metadata": {},
            "outputs": [],
            "execution_count": 3
        },
        {
            "cell_type": "markdown",
            "source": "### **Default settings**",
            "metadata": {}
        },
        {
            "cell_type": "code",
            "source": "mssqlctl_configuration_profile = 'aks-dev-test'\nmssqlctl_configuration_file = 'mssql-bdc-configuration'\n\n# Show parameter values\nprint('')\nprint(f'mssql_cluster_name = {mssql_cluster_name}')\nprint(f'docker_username = {docker_username}')\nprint(f'mssqlctl_configuration_profile = {mssqlctl_configuration_profile}')\nprint(f'mssqlctl_configuration_file = {mssqlctl_configuration_file}')\nprint(f'bdc_controller_username = {bdc_controller_username}')\nprint('')",
            "metadata": {},
            "outputs": [],
            "execution_count": 4
        },
        {
            "cell_type": "markdown",
            "source": "### **List the MSSQLCTL configuration profiles**",
            "metadata": {}
        },
        {
            "cell_type": "code",
            "source": "import os\nos.environ[\"ACCEPT_EULA\"] = 'yes'\ncmd = f'mssqlctl bdc config list'\nrun_command()",
            "metadata": {},
            "outputs": [],
            "execution_count": 5
        },
        {
            "cell_type": "markdown",
            "source": "### **Create a MSSQLCTL configuration file**",
            "metadata": {}
        },
        {
            "cell_type": "code",
            "source": "# Create a configuration file\ncmd = f'mssqlctl bdc config init --source {mssqlctl_configuration_profile} --target {mssqlctl_configuration_file} --force'\nrun_command()\n\n# Set the 'big data cluster' name\njsonPath = '\"metadata.name=''{0}''\"'.format(mssql_cluster_name)\ncmd = f'mssqlctl bdc config section set -c {mssqlctl_configuration_file} -j {jsonPath}'\nrun_command()",
            "metadata": {},
            "outputs": [],
            "execution_count": 6
        },
        {
            "cell_type": "markdown",
            "source": "### **Create SQL Server 2019 big data cluster**",
            "metadata": {}
        },
        {
            "cell_type": "code",
            "source": "import os\nprint (f'Creating SQL Server 2019 big data cluster: {mssql_cluster_name} using configuration file {mssqlctl_configuration_file}')\nos.environ[\"CONTROLLER_USERNAME\"] = bdc_controller_username\nos.environ[\"CONTROLLER_PASSWORD\"] = bdc_password\nos.environ[\"MSSQL_SA_PASSWORD\"] = bdc_password\nos.environ[\"KNOX_PASSWORD\"] = bdc_password\nos.environ[\"DOCKER_USERNAME\"] = docker_username\nos.environ[\"DOCKER_PASSWORD\"] = docker_password\n\ncmd = f'mssqlctl bdc create -c {mssqlctl_configuration_file} --accept-eula yes'\nrun_command()",
            "metadata": {},
            "outputs": [],
            "execution_count": 7
        },
        {
            "cell_type": "markdown",
            "source": "### **Login to SQL Server 2019 big data cluster**",
            "metadata": {}
        },
        {
            "cell_type": "code",
            "source": "cmd = f'mssqlctl login --cluster-name {mssql_cluster_name}'\nrun_command()",
            "metadata": {},
            "outputs": [],
            "execution_count": 8
        },
        {
            "cell_type": "markdown",
            "source": "### **Show SQL Server 2019 big data cluster endpoints**",
            "metadata": {}
        },
        {
            "cell_type": "code",
            "source": "import json\nimport pandas as pd\nfrom IPython.display import *\npd.set_option('display.max_colwidth', -1)\n\ndef formatColumnNames(column):\n    return ' '.join(word[0].upper() + word[1:] for word in column.split())\n\ndef show_results(results):\n    df = pd.DataFrame(results)\n    df.columns = [formatColumnNames(n) for n in results[0].keys()]\n    mydata = HTML(df.to_html(render_links=True))\n    display(mydata)\n\ncmd = f'mssqlctl bdc endpoint list'\nendpointsResults  = !{cmd}\nendpointsInJson = json.loads(''.join(endpointsResults))\nshow_results(endpointsInJson)",
            "metadata": {},
            "outputs": [],
            "execution_count": 9
        },
        {
            "cell_type": "markdown",
            "source": "### **Connect to master SQL Server instance in Azure Data Studio**\r\nClick the link below to connect to the master SQL Server instance of the SQL Server 2019 big data cluster.",
            "metadata": {}
        },
        {
            "cell_type": "code",
            "source": "filteredEndpoints = [x for x in endpointsInJson if x['name'] == 'sql-server-master']\r\nif filteredEndpoints and len(filteredEndpoints) == 1:\r\n    display(HTML(\"<h3><a href=\\\"command:azdata.connect?{&quot;serverName&quot;:&quot;\"+filteredEndpoints[0]['endpoint']+\"&quot;,&quot;providerName&quot;:&quot;MSSQL&quot;, &quot;authenticationType&quot;:&quot;SqlLogin&quot;,&quot;userName&quot;:&quot;sa&quot;,&quot;password&quot;:&quot;\"+bdc_password+\"&quot;}\\\">Click here to connect to SQL Server</a></h3>\"))",
            "metadata": {},
            "outputs": [],
            "execution_count": 10
        }
    ]
}