Cavonac/offmig (#13865)

* offline migration changes

updates to offline migration notebooks, toc, glossary, creating a new notebook

* variables table

* Update offline-mig readme

wording

* Update readme.md

* content

* overview and titles

for consistency, i updated the overview and titles

* Update readme.md

clarity and content

* removed create sql backups

* added term

* updating notebook

* adding code

* wording updates

* wording updates

* removed image no longer used

* avoid conflict with another PR
replaced with a notebook
This commit is contained in:
Christopher C
2021-01-25 17:31:43 -08:00
committed by GitHub
parent 728aeded01
commit 52fbdca2ed
8 changed files with 207 additions and 234 deletions

View File

@@ -2,7 +2,8 @@
"metadata": {
"kernelspec": {
"name": "powershell",
"display_name": "PowerShell"
"display_name": "PowerShell",
"language": "powershell"
},
"language_info": {
"name": "powershell",
@@ -17,33 +18,65 @@
{
"cell_type": "markdown",
"source": [
"Migrate SQL Server Instance to Azure SQL Server VM\n",
"================================"
"# Migrate SQL Server Database to Azure SQL DB\n",
"\n",
"## Overview\n",
"\n",
"Use this notebook when attempting to migrate a SQL Server database to Azure SQL Database. The process is different than using an Azure SQL Virtual Machine or Azure SQL Managed Instance. Refer to this [Microsoft docs](https://docs.microsoft.com/en-us/azure/dms/howto-sql-server-to-azure-sql-powershell) article for the method described.\n",
"\n",
"## Variables\n",
"\n",
"Use the table below as reference when configuring this notebook for execution.\n",
"\n",
"| Line | Variable | Description | Example |\n",
"| --- | --- | --- | --- |\n",
"| 1 | Subscription | Azure subscription name or ID | guid |\n",
"| 2 | dmsName | Name of the Data Migration Service | MyDms |\n",
"| 3 | dmsResourceGroup | Exact name of resource group to target or create | _MyAz\\_RG_ |\n",
"| 4 | loc | Specifies the location of the service | EastUS2 |\n",
"| 5 | vNetName | Specify the name of the vnet Azure resource to use | vnet1 |\n",
"| 6 | Subnet | Name of the Azure subnet resource to use | subnet1 |\n",
"| 7-8 | dmsSku | This parameter corresponds to DMS Sku name. The currently supported sku is listed. | GeneralPurpose\\_4vCores |\n",
"| 9 | TargetSqlServer | Target SQL server to migrate to | \"sqllazuretarget.database.windows.net\" |\n",
"| 10 | SourceDb | Name of database to migrate from, the _source_ | AdventureWorks2016 |\n",
"| 11 | TargetDb | Name of database to migrate to, in case it differs from the source, the _target_ | AdventureWorks2016 |\n",
"\n",
"Lastly, specify the table names explicitly as a PowerShell list of names:\n",
"\n",
"> `$Tables = @(`<span style=\"color: rgb(215, 186, 125); font-family: Menlo, Monaco, Consolas, &quot;Droid Sans Mono&quot;, &quot;Courier New&quot;, monospace, &quot;Droid Sans Fallback&quot;; font-size: 12px;\">\"HumanResources.Department\",&nbsp;</span> <span style=\"color: rgb(215, 186, 125); font-family: Menlo, Monaco, Consolas, &quot;Droid Sans Mono&quot;, &quot;Courier New&quot;, monospace, &quot;Droid Sans Fallback&quot;; font-size: 12px;\">\"HumanResources.Employee\",&nbsp;</span> <span style=\"color: rgb(215, 186, 125); font-family: Menlo, Monaco, Consolas, &quot;Droid Sans Mono&quot;, &quot;Courier New&quot;, monospace, &quot;Droid Sans Fallback&quot;; font-size: 12px;\">\"HumanResources.EmployeeDepartmentHistory\")</span>"
],
"metadata": {
"azdata_cell_guid": "f706da59-22c3-4317-bf41-c00dde794097"
}
},
{
"cell_type": "markdown",
"source": [
"### Load Required Modules\r\n",
""
],
"metadata": {
"azdata_cell_guid": "67e288c6-63df-475e-9cad-bab323d30c4e"
}
},
{
"cell_type": "code",
"source": [
"Import-Module dbatools\r\n",
"Import-Module Az.Resources\r\n",
"Import-Module Az.Storage"
"$Subscription = \"\"\r\n",
"$dmsName = \"\"\r\n",
"$dmsResourceGroup = \"\"\r\n",
"$loc = \"\"\r\n",
"$vNetName = \"\"\r\n",
"$Subnet = Subnet1\r\n",
"#$dmsSku = Basic_2vCores # unclear which is correct, listing both for now\r\n",
"$dmsSku = GeneralPurpose_4vCores \r\n",
"$TargetSqlServer = \"\"\r\n",
"$SourceDb = \"\"\r\n",
"$TargetDb = \"\"\r\n",
"\r\n",
"# Define a list of tables to migrate\r\n",
"$Tables = @()\r\n",
"<# :::For example::: \r\n",
"$Tables = @(\"HumanResources.Department\", `\r\n",
" \"HumanResources.Employee\", `\r\n",
" \"HumanResources.EmployeeDepartmentHistory\", `\r\n",
" \"HumanResources.EmployeePayHistory\", `\r\n",
" \"HumanResources.JobCandidate\", `\r\n",
" \"HumanResources.Shift\")\r\n",
"#>"
],
"metadata": {
"azdata_cell_guid": "26df73fa-6f4f-40b4-8c47-10ce7e2db404",
"tags": []
"azdata_cell_guid": "2624afc0-2403-4d42-ad88-6adcfe1a5c2b"
},
"outputs": [],
"execution_count": null
@@ -51,180 +84,34 @@
{
"cell_type": "markdown",
"source": [
"### Choose Migration Source\r\n",
"\r\n",
"Required parameters:\r\n",
"\r\n",
"- Server Name\r\n",
"- Database Name\r\n",
"- "
"## Connections"
],
"metadata": {
"azdata_cell_guid": "1e70b806-c94d-4be2-87cf-ad73fb85821d"
"azdata_cell_guid": "f0a5e313-a2a6-4e75-ab94-5d96436b8473"
}
},
{
"cell_type": "code",
"source": [
"$Credential = Get-Credential -Message \"Type the name and password of the local administrator account.\"\r\n",
"$Credential"
],
"metadata": {
"azdata_cell_guid": "41faae8f-6245-4acb-88d5-dbb0b92ad7f5"
},
"outputs": [],
"execution_count": null
},
{
"cell_type": "code",
"source": [
"# Variables\r\n",
"# Configure virtual network\r\n",
"$vNet = Get-AzVirtualNetwork -ResourceGroupName $dmsResourceGroup -Name $vNetName\r\n",
"\r\n",
"## Global\r\n",
"$Location = \"West US 2\"\r\n",
"$ResourceGroupName = \"sqlmig\"\r\n",
"$vSubNet = Get-AzVirtualNetworkSubnetConfig -VirtualNetwork $vNet -Name $Subnet\r\n",
"\r\n",
"## Storage\r\n",
"$StorageName = $ResourceGroupName + \"storage\"\r\n",
"$StorageSku = \"Premium_LRS\"\r\n",
"# Create a Database Connection Info object for the source connections\r\n",
"$sourceConnInfo = New-AzDmsConnInfo -ServerType SQL `\r\n",
" -DataSource $SourceSqlServer `\r\n",
" -AuthType SqlAuthentication `\r\n",
" -TrustServerCertificate:$true\r\n",
"\r\n",
"## Network\r\n",
"$InterfaceName = $ResourceGroupName + \"ServerInterface\"\r\n",
"$NsgName = $ResourceGroupName + \"nsg\"\r\n",
"$VNetName = $ResourceGroupName + \"VNet\"\r\n",
"$SubnetName = \"Default\"\r\n",
"$VNetAddressPrefix = \"10.0.0.0/16\"\r\n",
"$VNetSubnetAddressPrefix = \"10.0.0.0/24\"\r\n",
"$TCPIPAllocationMethod = \"Dynamic\"\r\n",
"$DomainName = $ResourceGroupName\r\n",
"\r\n",
"##Compute\r\n",
"$VMName = $ResourceGroupName + \"VM\"\r\n",
"$ComputerName = $ResourceGroupName + \"Server\"\r\n",
"$VMSize = \"Standard_DS13_v2\"\r\n",
"$OSDiskName = $VMName + \"OSDisk\"\r\n",
"\r\n",
"##Image\r\n",
"$PublisherName = \"MicrosoftSQLServer\"\r\n",
"$OfferName = \"SQL2017-WS2016\"\r\n",
"$Sku = \"SQLDEV\"\r\n",
"$Version = \"latest\"\r\n",
""
"# target connection\r\n",
"$targetConnInfo = New-AzDmsConnInfo -ServerType SQL `\r\n",
" -DataSource $TargetSqlServer `\r\n",
" -AuthType SqlAuthentication `\r\n",
" -TrustServerCertificate:$false"
],
"metadata": {
"azdata_cell_guid": "7e251e62-b47c-4800-986d-b71be8bc0a21"
},
"outputs": [],
"execution_count": null
},
{
"cell_type": "code",
"source": [
"# Resource Group\r\n",
"New-AzResourceGroup -Name $ResourceGroupName -Location $Location\r\n",
"$ResourceGroupName\r\n",
"$Location"
],
"metadata": {
"azdata_cell_guid": "eea4904c-55f0-47a4-81a3-5daf0a864687"
},
"outputs": [],
"execution_count": null
},
{
"cell_type": "code",
"source": [
"# Storage\r\n",
"$StorageAccount = New-AzStorageAccount -ResourceGroupName $ResourceGroupName -Name $StorageName -SkuName $StorageSku -Kind \"Storage\" -Location $Location\r\n",
""
],
"metadata": {
"azdata_cell_guid": "b19f1450-0283-4772-862a-3e59b65d0e75"
},
"outputs": [],
"execution_count": null
},
{
"cell_type": "code",
"source": [
"$StorageAccount = Get-AzStorageAccount -ResourceGroupName $ResourceGroupName -Name $StorageName"
],
"metadata": {
"azdata_cell_guid": "42602cef-76ed-4c91-a2f0-a7a8a6132eff"
},
"outputs": [],
"execution_count": null
},
{
"cell_type": "code",
"source": [
"# Network\r\n",
"$SubnetConfig = New-AzVirtualNetworkSubnetConfig -Name $SubnetName -AddressPrefix $VNetSubnetAddressPrefix\r\n",
"$VNet = New-AzVirtualNetwork -Name $VNetName -ResourceGroupName $ResourceGroupName -Location $Location -AddressPrefix $VNetAddressPrefix -Subnet $SubnetConfig\r\n",
"$PublicIp = New-AzPublicIpAddress -Name $InterfaceName -ResourceGroupName $ResourceGroupName -Location $Location -AllocationMethod $TCPIPAllocationMethod -DomainNameLabel $DomainName\r\n",
"#$NsgRuleRDP = New-AzNetworkSecurityRuleConfig -Name \"RDPRule\" -Protocol Tcp -Direction Inbound -Priority 1000 -SourceAddressPrefix * -SourcePortRange * -DestinationAddressPrefix * -DestinationPortRange 3389 -Access Allow\r\n",
"$NsgRuleSQL = New-AzNetworkSecurityRuleConfig -Name \"MSSQLRule\" -Protocol Tcp -Direction Inbound -Priority 1001 -SourceAddressPrefix * -SourcePortRange * -DestinationAddressPrefix * -DestinationPortRange 1433 -Access Allow\r\n",
"#$Nsg = New-AzNetworkSecurityGroup -ResourceGroupName $ResourceGroupName -Location $Location -Name $NsgName -SecurityRules $NsgRuleRDP,$NsgRuleSQL\r\n",
"$Nsg = New-AzNetworkSecurityGroup -ResourceGroupName $ResourceGroupName -Location $Location -Name $NsgName -SecurityRules $NsgRuleSQL\r\n",
"$Interface = New-AzNetworkInterface -Name $InterfaceName -ResourceGroupName $ResourceGroupName -Location $Location -SubnetId $VNet.Subnets[0].Id -PublicIpAddressId $PublicIp.Id -NetworkSecurityGroupId $Nsg.Id\r\n",
""
],
"metadata": {
"azdata_cell_guid": "0435fd83-3e9c-4929-930a-2b7022db3f99"
},
"outputs": [],
"execution_count": null
},
{
"cell_type": "code",
"source": [
"$StorageAccount"
],
"metadata": {
"azdata_cell_guid": "37e1869b-eb9f-465a-87d3-78120e4c6d06"
},
"outputs": [],
"execution_count": null
},
{
"cell_type": "code",
"source": [
"# Compute\r\n",
"$VirtualMachine = New-AzVMConfig -VMName $VMName -VMSize $VMSize\r\n",
"#$Credential = Get-Credential -Message \"Type the name and password of the local administrator account.\"\r\n",
"$VirtualMachine = Set-AzVMOperatingSystem -VM $VirtualMachine -Windows -ComputerName $ComputerName -Credential $Credential -ProvisionVMAgent -EnableAutoUpdate #-TimeZone = $TimeZone\r\n",
"$VirtualMachine = Add-AzVMNetworkInterface -VM $VirtualMachine -Id $Interface.Id\r\n",
"#$OSDiskUri = $StorageAccount.PrimaryEndpoints.Blob.ToString() + \"vhds/\" + $OSDiskName + \".vhd\"\r\n",
"#$VirtualMachine = Set-AzVMOSDisk -VM $VirtualMachine -Name $OSDiskName -VhdUri $OSDiskUri -Caching ReadOnly -CreateOption FromImage\r\n",
"$VirtualMachine = Set-AzVMSourceImage -VM $VirtualMachine -PublisherName $PublisherName -Offer $OfferName -Skus $Sku -Version $Version"
],
"metadata": {
"azdata_cell_guid": "d0a4da84-d591-436e-8b6c-d4cb2787a6e2"
},
"outputs": [],
"execution_count": null
},
{
"cell_type": "code",
"source": [
"# Create the VM in Azure\r\n",
"New-AzVM -ResourceGroupName $ResourceGroupName -Location $Location -VM $VirtualMachine\r\n",
""
],
"metadata": {
"azdata_cell_guid": "70967b75-1ef7-4d69-aad9-0184cfb44aa9"
},
"outputs": [],
"execution_count": null
},
{
"cell_type": "code",
"source": [
"# Add the SQL IaaS Extension, and choose the license type\r\n",
"New-AzSqlVM -ResourceGroupName $ResourceGroupName -Name $VMName -Location $Location -LicenseType \"PAYG\""
],
"metadata": {
"azdata_cell_guid": "a8db01b1-a009-4367-bd39-77187482afc3"
"azdata_cell_guid": "dbf40b4b-af83-46c6-9db1-15f45eb24382"
},
"outputs": [],
"execution_count": null
@@ -232,24 +119,33 @@
{
"cell_type": "markdown",
"source": [
"### Verify No Active Connections"
"## Select Data by Defining Database and Table Mapping"
],
"metadata": {
"azdata_cell_guid": "81259d7e-62ac-4cdd-9e1b-2cb4ddb3d3b2"
"azdata_cell_guid": "b9113be6-0aef-41fd-afe8-e1ce51c5999b"
}
},
{
"cell_type": "code",
"source": [
"# Create a list of databases to migrate\r\n",
"$dbInfo1 = New-AzDataMigrationDatabaseInfo -SourceDatabaseName $SourceDb\r\n",
"$dbList = @($dbInfo1)\r\n",
"\r\n",
"$ServerName = \"sqltools2017-3\"\r\n",
"$DatabaseName = \"Keep_WideWorldImporters\"\r\n",
"# Create a table map \r\n",
"$tableMap = New-Object 'system.collections.generic.dictionary[string,string]'\r\n",
"foreach ($table in $Tables)\r\n",
"{\r\n",
" $tableMap.Add($table,$table)\r\n",
"}\r\n",
"\r\n",
"Get-DbaProcess -SqlInstance $ServerName -Database $DatabaseName | \r\n",
"Select Host, login, Program"
"# Select the data\r\n",
"$selectedDbs = New-AzDmsSelectedDB -MigrateSqlServerSqlDb -Name $SourceDb `\r\n",
" -TargetDatabaseName $TargetDb `\r\n",
" -TableMap $tableMap"
],
"metadata": {
"azdata_cell_guid": "28393e59-4ea1-4f0f-8f9f-8a504f15e723"
"azdata_cell_guid": "3b6551e9-4d42-441e-8fe1-f247471995df"
},
"outputs": [],
"execution_count": null
@@ -257,19 +153,34 @@
{
"cell_type": "markdown",
"source": [
"### Create Target SQL Server VM"
"## Create Service Project"
],
"metadata": {
"azdata_cell_guid": "07d076d0-abf3-496c-8ecb-f85102c4104b"
"azdata_cell_guid": "a9abeeb9-4f45-4e0c-8b1b-20270dddece4"
}
},
{
"cell_type": "code",
"source": [
""
"$service = New-AzDms -ResourceGroupName $dmsResourceGroup `\r\n",
" -ServiceName $dmsName `\r\n",
" -Location $loc `\r\n",
" -Sku $dmsSku `\r\n",
" -VirtualSubnetId $vSubNet.Id\r\n",
"\r\n",
"# Create a project object\r\n",
"$project = New-AzDataMigrationProject -ResourceGroupName $dmsResourceGroup `\r\n",
" -ServiceName $service.Name `\r\n",
" -ProjectName $dmsNameProject `\r\n",
" -Location $loc `\r\n",
" -SourceType SQL `\r\n",
" -TargetType SQLDB `\r\n",
" -SourceConnection $sourceConnInfo `\r\n",
" -TargetConnection $targetConnInfo `\r\n",
" -DatabaseInfo $dbList"
],
"metadata": {
"azdata_cell_guid": "6f190c2d-7361-4db8-819d-29087eae8aaa"
"azdata_cell_guid": "77415bea-b098-4a9f-a166-9751e4261fc1"
},
"outputs": [],
"execution_count": null
@@ -277,60 +188,104 @@
{
"cell_type": "markdown",
"source": [
"### Create temporary resources for data movement"
"## Credentials for Source and Target"
],
"metadata": {
"azdata_cell_guid": "cc18027e-4636-465d-abaf-f3de88fea406"
"azdata_cell_guid": "f8c27b61-d2c9-43ee-bc93-8182752edb33"
}
},
{
"cell_type": "code",
"source": [
"$location = \"westus\"\r\n",
"$resourceGroup = \"temp-sqlmigration\"\r\n",
"$blobStorageAccount = \"temp-sqlmigration\"\r\n",
"$containerName = \"backups\"\r\n",
"\r\n",
"\r\n",
"\r\n",
"New-AzResourceGroup"
"$secpasswd = ConvertTo-SecureString -String $sourcePassword -AsPlainText -Force\r\n",
"$sourceCred = New-Object System.Management.Automation.PSCredential ($sourceUserName, $secpasswd)\r\n",
"$secpasswd = ConvertTo-SecureString -String $targetPassword -AsPlainText -Force\r\n",
"$targetCred = New-Object System.Management.Automation.PSCredential ($targetUserName, $secpasswd)"
],
"metadata": {
"azdata_cell_guid": "f7d53cb1-a55d-4634-95f7-d3e8cf9fab52"
"azdata_cell_guid": "61aa72d1-7492-41e8-847b-89a76ca28e0f"
},
"outputs": [],
"execution_count": null
},
{
"cell_type": "code",
"cell_type": "markdown",
"source": [
"$targetLogin = Get-Credential -Message \"Login to target SQL Server instance as:\""
"## Create the Migration Task\r\n",
"Uncomment lines to perform validation checks during the task."
],
"metadata": {
"azdata_cell_guid": "c3dbd1a7-5514-4fdc-9430-736c92e875a4"
"azdata_cell_guid": "681feaa0-f7d8-4322-af0c-75ab6402fd6e"
}
},
{
"cell_type": "code",
"source": [
"$migTask = New-AzDataMigrationTask -TaskType MigrateSqlServerSqlDb `\r\n",
" -ResourceGroupName $dmsResourceGroup `\r\n",
" -ServiceName $service.Name `\r\n",
" -ProjectName $project.Name `\r\n",
" -TaskName $dmsNameTask `\r\n",
" -SourceConnection $sourceConnInfo `\r\n",
" -SourceCred $sourceCred `\r\n",
" -TargetConnection $targetConnInfo `\r\n",
" -TargetCred $targetCred `\r\n",
" -SelectedDatabase $selectedDbs `\r\n",
"# -SchemaValidation `\r\n",
"# -DataIntegrityValidation `\r\n",
"# -QueryAnalysisValidation `"
],
"metadata": {
"azdata_cell_guid": "4dc629cc-1978-4bf8-80fc-69de093ac435"
},
"outputs": [],
"execution_count": null
},
{
"cell_type": "code",
"cell_type": "markdown",
"source": [
"$myGwIp = Get-AzPublicIpAddress -Name $GwIP1 -ResourceGroup $RG1\r\n",
"$myGwIp.IpAddress"
"## Monitor the Migration Task"
],
"metadata": {
"azdata_cell_guid": "bc6e8330-95bb-44a5-a117-020f657cad2b"
"azdata_cell_guid": "9074e9e5-f5f7-4d30-a4ec-b6d3f112a600"
}
},
{
"cell_type": "code",
"source": [
"# Monitor the migration\r\n",
"if (($migTask.ProjectTask.Properties.State -eq \"Running\") -or ($migTask.ProjectTask.Properties.State -eq \"Queued\"))\r\n",
"{\r\n",
" write-host \"migration task running\"\r\n",
"}\r\n",
"else \r\n",
"{\r\n",
" Write-Host \"migration task is $migTask.ProjectTask.Properties.State\"\r\n",
"}"
],
"metadata": {
"azdata_cell_guid": "89af76b7-1801-4d26-b6cd-907a9b455839"
},
"outputs": [],
"execution_count": null
},
{
"cell_type": "code",
"cell_type": "markdown",
"source": [
""
"## Deleting the DMS Instance\r\n",
"This should remove the DMS instance from the Resource Group. Uncomment to perform the task. "
],
"metadata": {
"azdata_cell_guid": "a61ffe69-0929-4246-8ad0-846f540f4e0c"
"azdata_cell_guid": "ed3bf1e7-a92f-4985-b89c-85f45b8e6821"
}
},
{
"cell_type": "code",
"source": [
"#Remove-AzDms -ResourceGroupName $dmsResourceGroup -ServiceName $dmsName"
],
"metadata": {
"azdata_cell_guid": "245b4d4e-7559-4534-bff8-3af390ac1e3a"
},
"outputs": [],
"execution_count": null