ckaczor/azuredatastudio
1
0
Fork 0
mirror of https://github.com/ckaczor/azuredatastudio.git synced 2026-02-16 18:46:40 -05:00
Code Issues Packages Projects Releases Wiki Activity

Changing Postgres Hyperscale to Postgres Azure Arc in ADS (#20445)

Browse Source 
* Hard code enginer ver to 14 and remove extension from wizard field and notebook

* Removed hyperscale text from everywhere but links

* Made PostgreSQL into PostgreSQL server and other text changes. Removed worker nodes

* Removed coordinator node text

* Removed worker nodes from compute + storage. Changed az arc-server to server-arc. postgresParameters.ts may need

* Removed tabs for extensions, coord nodes, worker nodes, resource health, diagnose and solve

* Removed node configuation and pg version from overview and properties. Edited arialabels

* Change server group nodes to server

* Removed type from overview table and aded Preview tags

* Removed info bubble and added padding to top of title

* Removed checkbox from PG create wizard. Added link and text

* Remove engine version

* Changed naming conventions

* Removed coord and server group from variables

* Removed properties

* Removed Server table, admin username, reset password

* Removed function from notebook

* Change wording

* bump version

Co-authored-by: Candice Ye <canye@microsoft.com>
This commit is contained in:
Candice Ye
2022-08-26 13:24:48 -07:00
committed by GitHub
parent 04ca5fbe42
commit bd0c4cdb51
26 changed files with 307 additions and 936 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
extensions/arc/notebooks/arcDataServices/_data/toc.yml
View File

@@ -6,5 +6,5 @@
url: /postgres/readme
not_numbered: true
sections:
- title: TSG100 - The Azure Arc-enabled PostgreSQL Hyperscale troubleshooter
- title: TSG100 - The Azure Arc-enabled PostgreSQL server troubleshooter
url: postgres/tsg100-troubleshoot-postgres

2
extensions/arc/notebooks/arcDataServices/content/postgres/readme.md
View File

@@ -8,4 +8,4 @@
## Notebooks in this Chapter
- [TSG100 - The Azure Arc-enabled PostgreSQL Hyperscale troubleshooter](../postgres/tsg100-troubleshoot-postgres.ipynb)
- [TSG100 - The Azure Arc-enabled PostgreSQL server troubleshooter](../postgres/tsg100-troubleshoot-postgres.ipynb)

128
extensions/arc/notebooks/arcDataServices/content/postgres/tsg100-troubleshoot-postgres.ipynb
View File

@@ -2,20 +2,17 @@
"cells": [
{
"cell_type": "markdown",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"TSG100 - The Azure Arc-enabled PostgreSQL Hyperscale troubleshooter\n",
"TSG100 - The Azure Arc-enabled PostgreSQL server troubleshooter\n",
"===================================================================\n",
"\n",
"Description\n",
"-----------\n",
"\n",
"Follow these steps to troubleshoot an Azure Arc-enabled PostgreSQL\n",
"Hyperscale Server.\n",
"Follow these steps to troubleshoot an Azure Arc-enabled PostgreSQL Server.\n",
"\n",
"Steps\n",
"-----\n",
@@ -29,7 +26,10 @@
"metadata": {
"tags": [
"parameters"
]
],
"vscode": {
"languageId": "python"
}
},
"outputs": [],
"source": [
@@ -45,11 +45,9 @@
},
{
"cell_type": "markdown",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"### Common functions\n",
"\n",
@@ -62,7 +60,10 @@
"metadata": {
"tags": [
"hide_input"
]
],
"vscode": {
"languageId": "python"
}
},
"outputs": [],
"source": [
@@ -279,7 +280,7 @@
" if user_provided_exe_name in retry_hints:\n",
" for retry_hint in retry_hints[user_provided_exe_name]:\n",
" if line_decoded.find(retry_hint) != -1:\n",
" if retry_count \u003c MAX_RETRIES:\n",
" if retry_count < MAX_RETRIES:\n",
" print(f\"RETRY: {retry_count} (due to: {retry_hint})\")\n",
" retry_count = retry_count + 1\n",
" output = run(cmd, return_output=return_output, retry_count=retry_count)\n",
@@ -326,11 +327,9 @@
},
{
"cell_type": "markdown",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"### Get Postgres server"
]
@@ -341,7 +340,10 @@
"metadata": {
"tags": [
"hide_input"
]
],
"vscode": {
"languageId": "python"
}
},
"outputs": [],
"source": [
@@ -364,7 +366,7 @@
"\n",
" pad = math.floor(math.log10(len(servers)) + 1) + 3\n",
" for i, s in enumerate(servers):\n",
" print(f'{f\"[{i+1}]\":\u003c{pad}}{full_name(s)}')\n",
" print(f'{f\"[{i+1}]\":<{pad}}{full_name(s)}')\n",
"\n",
" while True:\n",
" try:\n",
@@ -372,7 +374,7 @@
" except ValueError:\n",
" continue\n",
"\n",
" if i \u003e= 1 and i \u003c= len(servers):\n",
" if i >= 1 and i <= len(servers):\n",
" server = servers[i-1]\n",
" namespace = server['metadata']['namespace']\n",
" name = server['metadata']['name']\n",
@@ -383,11 +385,9 @@
},
{
"cell_type": "markdown",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"### Summarize all resources"
]
@@ -396,7 +396,10 @@
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
"tags": [],
"vscode": {
"languageId": "python"
}
},
"outputs": [],
"source": [
@@ -411,11 +414,9 @@
},
{
"cell_type": "markdown",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"### Troubleshoot the server"
]
@@ -424,7 +425,10 @@
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
"tags": [],
"vscode": {
"languageId": "python"
}
},
"outputs": [],
"source": [
@@ -434,11 +438,9 @@
},
{
"cell_type": "markdown",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"### Troubleshoot the pods"
]
@@ -447,7 +449,10 @@
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
"tags": [],
"vscode": {
"languageId": "python"
}
},
"outputs": [],
"source": [
@@ -471,11 +476,9 @@
},
{
"cell_type": "markdown",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"### Troubleshoot the containers"
]
@@ -484,7 +487,10 @@
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
"tags": [],
"vscode": {
"languageId": "python"
}
},
"outputs": [],
"source": [
@@ -493,7 +499,7 @@
" pod_name = pod['metadata']['name']\n",
" cons = pod['spec']['containers']\n",
" con_statuses = pod['status'].get('containerStatuses', [])\n",
" display(Markdown(f'#### Troubleshooting {len(cons)} container{\"\" if len(cons) \u003c 2 else \"s\"} '\n",
" display(Markdown(f'#### Troubleshooting {len(cons)} container{\"\" if len(cons) < 2 else \"s\"} '\n",
" f'containers for pod {namespace}.{pod_name}'))\n",
"\n",
" for i, con in enumerate(cons):\n",
@@ -509,18 +515,16 @@
" run(f'kubectl logs -n {namespace} {pod_name} {con_name} --tail {tail_lines}')\n",
"\n",
" # Get logs from the previous terminated container if one exists\n",
" if con_restarts \u003e 0:\n",
" if con_restarts > 0:\n",
" display(Markdown(f'#### Logs from previous terminated container {namespace}.{pod_name}/{con_name}'))\n",
" run(f'kubectl logs -n {namespace} {pod_name} {con_name} --tail {tail_lines} --previous')"
]
},
{
"cell_type": "markdown",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"### Troubleshoot the PersistentVolumeClaims"
]
@@ -529,7 +533,10 @@
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
"tags": [],
"vscode": {
"languageId": "python"
}
},
"outputs": [],
"source": [
@@ -541,7 +548,10 @@
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
"tags": [],
"vscode": {
"languageId": "python"
}
},
"outputs": [],
"source": [
@@ -549,33 +559,10 @@
]
}
],
"nbformat": 4,
"nbformat_minor": 5,
"metadata": {
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"pansop": {
"related": "",
"test": {
"strategy": "",
"types": null,
"disable": {
"reason": "",
"workitems": null,
"types": null
}
},
"target": {
"current": "public",
"final": "public"
},
"internal": {
"parameters": null,
"symlink": false
},
"timeout": "0"
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"codemirror_mode": "{ Name: \"\", Version: \"\"}",
@@ -586,6 +573,29 @@
"pygments_lexer": "",
"version": ""
},
"pansop": {
"internal": {
"parameters": null,
"symlink": false
},
"related": "",
"target": {
"current": "public",
"final": "public"
},
"test": {
"disable": {
"reason": "",
"types": null,
"workitems": null
},
"strategy": "",
"types": null
},
"timeout": "0"
},
"widgets": []
}
},
"nbformat": 4,
"nbformat_minor": 5
}

163
extensions/arc/notebooks/arcDeployment/deploy.postgres.existing.arc.ipynb
View File

@@ -1,46 +1,28 @@
{
"metadata": {
"kernelspec": {
"name": "python3",
"display_name": "Python 3",
"language": "python"
},
"language_info": {
"name": "python",
"version": "3.6.6",
"mimetype": "text/x-python",
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"pygments_lexer": "ipython3",
"nbconvert_exporter": "python",
"file_extension": ".py"
}
},
"nbformat_minor": 2,
"nbformat": 4,
"cells": [
{
"cell_type": "markdown",
"metadata": {
"azdata_cell_guid": "e4ed0892-7b5a-4d95-bd0d-a6c3eb0b2c99"
},
"source": [
"![Microsoft](https://raw.githubusercontent.com/microsoft/azuredatastudio/main/extensions/arc/images/microsoft-small-logo.png)\n",
" \n",
"## Create a PostgreSQL Hyperscale - Azure Arc on an existing Azure Arc Data Controller\n",
"## Create a PostgreSQL server - Azure Arc on an existing Azure Arc Data Controller\n",
" \n",
"This notebook walks through the process of creating a PostgreSQL Hyperscale - Azure Arc on an existing Azure Arc Data Controller.\n",
"This notebook walks through the process of creating a PostgreSQL server - Azure Arc on an existing Azure Arc Data Controller.\n",
" \n",
"* Follow the instructions in the **Prerequisites** cell to install the tools if not already installed.\n",
"* Make sure you have the target Azure Arc Data Controller already created.\n",
"\n",
"<span style=\"color:red\"><font size=\"3\">Please press the \"Run All\" button to run the notebook</font></span>"
],
"metadata": {
"azdata_cell_guid": "e4ed0892-7b5a-4d95-bd0d-a6c3eb0b2c99"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"azdata_cell_guid": "20fe3985-a01e-461c-bce0-235f7606cc3c"
},
"source": [
"### **Prerequisites** \n",
"Ensure the following tools are installed and added to PATH before proceeding.\n",
@@ -49,23 +31,25 @@
"|---|---|---|\n",
"|Azure CLI (az) | Command-line tool for installing and managing resources in an Azure Arc cluster |[Installation](https://docs.microsoft.com/cli/azure/install-azure-cli-windows?tabs=azure-cli) |\n",
"|Azure CLI arcdata extension | Commands for using Azure Arc for Azure data services. | [Installation](https://docs.microsoft.com/azure/azure-arc/data/install-arcdata-extension)"
],
"metadata": {
"azdata_cell_guid": "20fe3985-a01e-461c-bce0-235f7606cc3c"
}
]
},
{
"cell_type": "markdown",
"source": [
"### **Setup and Check Prerequisites**"
],
"metadata": {
"azdata_cell_guid": "68531b91-ddce-47d7-a1d8-2ddc3d17f3e7"
}
},
"source": [
"### **Setup and Check Prerequisites**"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"azdata_cell_guid": "749d8dba-3da8-46e9-ae48-2b38056ab7a2",
"tags": []
},
"outputs": [],
"source": [
"import sys,os,json,subprocess\n",
"def run_command():\n",
@@ -80,81 +64,74 @@
" return output.stdout.decode(\"utf-8\")\n",
"cmd = 'az --version'\n",
"out = run_command()\n"
],
"outputs": [],
"metadata": {
"azdata_cell_guid": "749d8dba-3da8-46e9-ae48-2b38056ab7a2",
"tags": []
}
]
},
{
"cell_type": "markdown",
"metadata": {
"azdata_cell_guid": "68ec0760-27d1-4ded-9a9f-89077c40b8bb"
},
"source": [
"### **Set variables**\n",
"\n",
"#### \n",
"\n",
"Generated by Azure Data Studio using the values collected in the 'Deploy PostgreSQL Hyperscale - Azure Arc instance' wizard"
],
"metadata": {
"azdata_cell_guid": "68ec0760-27d1-4ded-9a9f-89077c40b8bb"
}
"Generated by Azure Data Studio using the values collected in the 'Deploy a PostgreSQL server - Azure Arc instance' wizard"
]
},
{
"cell_type": "markdown",
"source": [
"### **Creating the PostgreSQL Hyperscale - Azure Arc instance**"
],
"metadata": {
"azdata_cell_guid": "90b0e162-2987-463f-9ce6-12dda1267189"
}
},
"source": [
"### **Creating the PostgreSQL server - Azure Arc instance**"
]
},
{
"cell_type": "code",
"execution_count": null,
"source": [
"print (f'Creating the PostgreSQL Hyperscale - Azure Arc instance')\n",
"\n",
"workers = f' --workers {postgres_server_group_workers}' if postgres_server_group_workers else \"\"\n",
"port = f' --port \"{postgres_server_group_port}\"' if postgres_server_group_port else \"\"\n",
"engine_version = f' --engine-version {postgres_server_group_engine_version}' if postgres_server_group_engine_version else \"\"\n",
"extensions = f' --extensions \"{postgres_server_group_extensions}\"' if postgres_server_group_extensions else \"\"\n",
"volume_size_data = f' --volume-size-data {postgres_server_group_volume_size_data}Gi' if postgres_server_group_volume_size_data else \"\"\n",
"volume_size_logs = f' --volume-size-logs {postgres_server_group_volume_size_logs}Gi' if postgres_server_group_volume_size_logs else \"\"\n",
"volume_size_backups = f' --volume-size-backups {postgres_server_group_volume_size_backups}Gi' if postgres_server_group_volume_size_backups else \"\"\n",
"\n",
"def get_per_role_argument(argument, roles, unit=''):\n",
" value = ','.join(f'{role}={value}{unit}' for role,value in roles.items() if value not in (None, ''))\n",
" return f' {argument} {value}' if value else ''\n",
"\n",
"cores_request = get_per_role_argument('--cores-request', {\n",
" 'c': postgres_server_group_coordinator_cores_request,\n",
" 'w': postgres_server_group_workers_cores_request\n",
"})\n",
"\n",
"cores_limit = get_per_role_argument('--cores-limit', {\n",
" 'c': postgres_server_group_coordinator_cores_limit,\n",
" 'w': postgres_server_group_workers_cores_limit\n",
"})\n",
"\n",
"memory_request = get_per_role_argument('--memory-request', {\n",
" 'c': postgres_server_group_coordinator_memory_request,\n",
" 'w': postgres_server_group_workers_memory_request\n",
"}, 'Gi')\n",
"\n",
"memory_limit = get_per_role_argument('--memory-limit', {\n",
" 'c': postgres_server_group_coordinator_memory_limit,\n",
" 'w': postgres_server_group_workers_memory_limit\n",
"}, 'Gi')\n",
"\n",
"os.environ[\"AZDATA_PASSWORD\"] = os.environ[\"AZDATA_NB_VAR_POSTGRES_SERVER_GROUP_PASSWORD\"]\n",
"cmd = f'az postgres arc-server create --name {postgres_server_group_name} --k8s-namespace {arc_data_controller_namespace} --use-k8s --storage-class-data {postgres_storage_class_data} --storage-class-logs {postgres_storage_class_logs} --storage-class-backups {postgres_storage_class_backups}{workers}{port}{engine_version}{extensions}{volume_size_data}{volume_size_logs}{volume_size_backups}{cores_request}{cores_limit}{memory_request}{memory_limit}'\n",
"out=run_command()"
],
"outputs": [],
"metadata": {
"azdata_cell_guid": "4fbaf071-55a1-40bc-be7e-7b9b5547b886"
}
},
"outputs": [],
"source": [
"print (f'Creating the PostgreSQL server - Azure Arc instance')\n",
"\n",
"port = f' --port \"{postgres_server_port}\"' if postgres_server_port else \"\"\n",
"volume_size_data = f' --volume-size-data {postgres_server_volume_size_data}Gi' if postgres_server_volume_size_data else \"\"\n",
"volume_size_logs = f' --volume-size-logs {postgres_server_volume_size_logs}Gi' if postgres_server_volume_size_logs else \"\"\n",
"\n",
"cores_request = f' --cores-request {postgres_server_cores_request}' if postgres_server_cores_request else \"\"\n",
"cores_limit = f' --cores-limit {postgres_server_cores_limit}' if postgres_server_cores_limit else \"\"\n",
"memory_request = f' --memory-request {postgres_server_memory_request}Gi' if postgres_server_memory_request else \"\"\n",
"memory_limit = f' --memory-limit {postgres_server_memory_limit}Gi' if postgres_server_memory_limit else \"\"\n",
"\n",
"os.environ[\"AZDATA_PASSWORD\"] = os.environ[\"AZDATA_NB_VAR_POSTGRES_SERVER_PASSWORD\"]\n",
"cmd = f'az postgres server-arc create --name {postgres_server_name} --k8s-namespace {arc_data_controller_namespace} --use-k8s --storage-class-data {postgres_storage_class_data} --storage-class-logs {postgres_storage_class_logs}{port}{volume_size_data}{volume_size_logs}{cores_request}{cores_limit}{memory_request}{memory_limit}'\n",
"out=run_command()"
]
}
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.6"
}
},
"nbformat": 4,
"nbformat_minor": 2
}