azuredatastudio/extensions/big-data-cluster/notebooks/deployment/2019/deploy-bdc-existing-aro.ipynb

Deploy SQL Server 2019 Big Data Cluster on an existing Azure Red Hat OpenShift cluster

This notebook walks through the process of deploying a SQL Server 2019 Big Data Cluster on an existing Azure Red Hat OpenShift cluster.

• Follow the instructions in the Prerequisites cell to install the tools if not already installed.
• The Required information will check and prompt you for password if it is not set in the environment variable. The password can be used to access the cluster controller, SQL Server, and Knox.

Please press the "Run all" button to run the notebook

Prerequisites

Ensure the following tools are installed and added to PATH before proceeding.

Tools	Description	Installation
kubectl	Command-line tool for monitoring the underlying Kubernetes cluster	Installation
azdata	Command-line tool for installing and managing a Big Data Cluster	Installation

Setup

In [ ]:

import pandas,sys,os,json,html,getpass,time
pandas_version = pandas.__version__.split('.')
pandas_major = int(pandas_version[0])
pandas_minor = int(pandas_version[1])
pandas_patch = int(pandas_version[2])
if not (pandas_major > 0 or (pandas_major == 0 and pandas_minor > 24) or (pandas_major == 0 and pandas_minor == 24 and pandas_patch >= 2)):
    sys.exit('Please upgrade the Notebook dependency before you can proceed, you can do it by running the "Reinstall Notebook dependencies" command in command palette (View menu -> Command Palette…).')
def run_command(command):
    print("Executing: " + command)
    !{command}
    if _exit_code != 0:
        sys.exit(f'Command execution failed with exit code: {str(_exit_code)}.\n\t{command}\n')
    print(f'Successfully executed: {command}')

Set variables

Generated by Azure Data Studio using the values collected in the Deploy Big Data Cluster wizard

Check dependencies

In [ ]:

run_command('kubectl version --client=true')
run_command('azdata --version')

Required information

In [ ]:

invoked_by_wizard = "AZDATA_NB_VAR_BDC_ADMIN_PASSWORD" in os.environ
if invoked_by_wizard:
    mssql_password = os.environ["AZDATA_NB_VAR_BDC_ADMIN_PASSWORD"]
    if mssql_auth_mode == "ad":
        mssql_domain_service_account_password = os.environ["AZDATA_NB_VAR_BDC_AD_DOMAIN_SVC_PASSWORD"]
else:
    mssql_password = getpass.getpass(prompt = 'SQL Server 2019 Big Data Cluster controller password')
    if mssql_password == "":
        sys.exit(f'Password is required.')
    confirm_password = getpass.getpass(prompt = 'Confirm password')
    if mssql_password != confirm_password:
        sys.exit(f'Passwords do not match.')
    if mssql_auth_mode == "ad":
        mssql_domain_service_account_password = getpass.getpass(prompt = 'Domain service account password')
        if mssql_domain_service_account_password == "":
            sys.exit(f'Domain service account password is required.')
print('You can also use the controller password to access Knox and SQL Server.')

Set and show current context

In [ ]:

run_command(f'kubectl config use-context {mssql_cluster_context}')
run_command('kubectl config current-context')

Make sure the target namespace already exists

In [ ]:

run_command(f'kubectl get namespace {mssql_cluster_name}')

Create deployment configuration files

In [ ]:

mssql_target_profile = 'ads-bdc-custom-profile'
if not os.path.exists(mssql_target_profile):
    os.mkdir(mssql_target_profile)
bdcJsonObj = json.loads(bdc_json)
controlJsonObj = json.loads(control_json)
bdcJsonFile = open(f'{mssql_target_profile}/bdc.json', 'w')
bdcJsonFile.write(json.dumps(bdcJsonObj, indent = 4))
bdcJsonFile.close()
controlJsonFile = open(f'{mssql_target_profile}/control.json', 'w')
controlJsonFile.write(json.dumps(controlJsonObj, indent = 4))
controlJsonFile.close()
print(f'Created deployment configuration folder: {mssql_target_profile}')

Create SQL Server 2019 Big Data Cluster

In [ ]:

print (f'Creating SQL Server 2019 Big Data Cluster: {mssql_cluster_name} using configuration {mssql_target_profile}')
os.environ["ACCEPT_EULA"] = 'yes'
os.environ["AZDATA_USERNAME"] = mssql_username
os.environ["AZDATA_PASSWORD"] = mssql_password
if mssql_auth_mode == "ad":
    os.environ["DOMAIN_SERVICE_ACCOUNT_USERNAME"] = mssql_domain_service_account_username
    os.environ["DOMAIN_SERVICE_ACCOUNT_PASSWORD"] = mssql_domain_service_account_password
if os.name == 'nt':
    print(f'If you don\'t see output produced by azdata, you can run the following command in a terminal window to check the deployment status:\n\t{os.environ["AZDATA_NB_VAR_KUBECTL"]} get pods -n {mssql_cluster_name} ')
run_command(f'azdata bdc create -c {mssql_target_profile}')

Login to SQL Server 2019 Big Data Cluster

In [ ]:

run_command(f'azdata login --namespace {mssql_cluster_name}')

Show SQL Server 2019 Big Data Cluster endpoints

In [ ]:

from IPython.display import *
pandas.set_option('display.max_colwidth', -1)
cmd = f'azdata bdc endpoint list'
cmdOutput = !{cmd}
endpoints = json.loads(''.join(cmdOutput))
endpointsDataFrame = pandas.DataFrame(endpoints)
endpointsDataFrame.columns = [' '.join(word[0].upper() + word[1:] for word in columnName.split()) for columnName in endpoints[0].keys()]
display(HTML(endpointsDataFrame.to_html(index=False, render_links=True)))

Connect to SQL Server Master instance in Azure Data Studio

Click the link below to connect to the SQL Server Master instance of the SQL Server 2019 Big Data Cluster.

In [ ]:

sqlEndpoints = [x for x in endpoints if x['name'] == 'sql-server-master']
if sqlEndpoints and len(sqlEndpoints) == 1:
    connectionParameter = '{"serverName":"' + sqlEndpoints[0]['endpoint'] + '","providerName":"MSSQL","authenticationType":"SqlLogin","userName":' + json.dumps(mssql_username) + ',"password":' + json.dumps(mssql_password) + '}'
    display(HTML('<br/><a href="command:azdata.connect?' + html.escape(connectionParameter)+'"><font size="3">Click here to connect to SQL Server Master instance</font></a><br/>'))
    display(HTML('<br/><span style="color:red"><font size="2">NOTE: The SQL Server password is included in this link, you may want to clear the results of this code cell before saving the notebook.</font></span>'))
else:
    sys.exit('Could not find the SQL Server Master instance endpoint.')