Azure SQL数据库：使用“复制活动”将表从一个Azure SQL数据库传输/迁移到另一个Azure SQL数据库

Question

我正在尝试将表从数据库A迁移到同一Azure SQL Server中的数据库B.我使用Azure数据工厂成功地将数据从Azure blob存储迁移到Azure数据库，但我似乎不明白如何修改我的ADF代码以跨数据库迁移数据。

我已经学会了通过This StackOverflow Link的帮助迁移数据但是我正在寻找使用Copy Activity迁移表。

Azure Blob到Azure SQL的管道如下所示。请建议可能导致数据从一个Azure SQL DB迁移到另一个Azure SQL DB的修改。这是完整的数据工厂代码。我正在寻找一种解决方法或至少一些可以指导我的资源。提前致谢。

Azure SQL链接服务

{
"name": "AzureSqlLinkedService",
"properties": {
    "description": "",
    "hubName": "dalete_hub",
    "type": "AzureSqlDatabase",
    "typeProperties": {
        "connectionString": "Data Source=tcp:server.database.windows.net,1433;Initial Catalog=DB;Integrated Security=False;User [email protected];Password=**********;Connect Timeout=30;Encrypt=True"
    }
}

}

Azure存储链接服务

{
"name": "AzureStorageLinkedService",
"properties": {
    "description": "",
    "hubName": "dalete_hub",
    "type": "AzureStorage",
    "typeProperties": {
        "connectionString": "DefaultEndpointsProtocol=https;AccountName=newstorageaccount;AccountKey=**********"
    }
}

}

输入数据集

   {
    "name": "InputDataset",
    "properties": {
        "structure": [
            {
                "name": "Region",
                "type": "String"
            },
            {
                "name": "Sales",
"type": "String"
            }
        ],
        "published": false,
        "type": "AzureBlob",
        "linkedServiceName": "AzureStorageLinkedService",
        "typeProperties": {
            "fileName": "data.txt",
            "folderPath": "adfpoc/",
            "format": {"type": "TextFormat",
                "columnDelimiter": ","
            }
        },
        "availability": {
            "frequency": "Hour",
            "interval": 1
        },
        "external": true,
        "policy": {}
    }
}

输出数据集

    {
    "name": "OutputDataset",
    "properties": {
        "structure": [
            {
                "name": "Region",
                "type": "String"
            },
            {
                "name": "Sales",
                "type": "String"
            }
        ],
        "published": false,
        "type": "AzureSqlTable",
        "linkedServiceName": "AzureSqlLinkedService",
        "typeProperties": {
            "tableName": "data"
        },
        "availability": {
            "frequency": "Hour",
            "interval": 1
        }
    }
}

ADF管道

  {
    "name": "ADFTutorialPipeline",
    "properties": {
        "description": "Copy data from a blob to Azure SQL table",
        "activities": [
            {
                "type": "Copy",
                "typeProperties": {
                    "source": {
                        "type": "BlobSource" },
                    "sink": {
                        "type": "SqlSink",
                        "writeBatchSize": 10000,
                        "writeBatchTimeout": "60.00:00:00"
                    }
                },
                "inputs": [
                    {
                        "name": "InputDataset"}
                ],
                "outputs": [
                    {
                        "name": "OutputDataset"
                    }
                ],
                "policy": {
                    "timeout": "01:00:00",
                    "concurrency": 1,
                    "executionPriorityOrder": "NewestFirst"
                },
                "scheduler": {
                    "frequency": "Hour",
                    "interval": 1
                },
                "name": "CopyFromBlobToSQL"
            }
        ],
        "start": "2019-03-11T00:00:00Z",
        "end": "2019-03-12T00:00:00Z",
        "isPaused": false,
        "hubName": "dalete_hub",
        "pipelineMode": "Scheduled"
    }
}

Answer 1

我和你一起尝试了同样的操作，我在Data Factory中成功地迁移了我的表。

例如，我在数据库table1中有一个表dbleon，我想将table1迁移到另一个带有复制活动的数据库dbleon1。

我在table1创建了一个新表dbmeon1，它与table1中的dbleon具有相同的模式。

这是我的ADF代码：

{
    "name": "CopyPipeline_0oh",
    "properties": {
        "activities": [
            {
                "name": "Copy_0oh",
                "type": "Copy",
                "policy": {
                    "timeout": "7.00:00:00",
                    "retry": 0,
                    "retryIntervalInSeconds": 30,
                    "secureOutput": false,
                    "secureInput": false
                },
                "userProperties": [
                    {
                        "name": "Source",
                        "value": "[dbo].[table1]"
                    },
                    {
                        "name": "Destination",
                        "value": "[dbo].[table1]"
                    }
                ],
                "typeProperties": {
                    "source": {
                        "type": "SqlSource"
                    },
                    "sink": {
                        "type": "SqlSink",
                        "writeBatchSize": 10000
                    },
                    "enableStaging": false,
                    "translator": {
                        "type": "TabularTranslator",
                        "columnMappings": {
                            "id": "id",
                            "name": "name"
                        }
                    }
                },
                "inputs": [
                    {
                        "referenceName": "SourceDataset_0oh",
                        "type": "DatasetReference"
                    }
                ],
                "outputs": [
                    {
                        "referenceName": "DestinationDataset_0oh",
                        "type": "DatasetReference"
                    }
                ]
            }
        ]
    },
    "type": "Microsoft.DataFactory/factories/pipelines"
}

来源数据集：

{
    "name": "SourceDataset_0oh",
    "properties": {
        "linkedServiceName": {
            "referenceName": "AzureSqlDatabase1",
            "type": "LinkedServiceReference"
        },
        "type": "AzureSqlTable",
        "structure": [
            {
                "name": "id",
                "type": "Int32"
            },
            {
                "name": "name",
                "type": "String"
            }
        ],
        "typeProperties": {
            "tableName": "[dbo].[table1]"
        }
    },
    "type": "Microsoft.DataFactory/factories/datasets"
}

目的地数据集：

{
    "name": "DestinationDataset_0oh",
    "properties": {
        "linkedServiceName": {
            "referenceName": "AzureSqlDatabase2",
            "type": "LinkedServiceReference"
        },
        "type": "AzureSqlTable",
        "structure": [
            {
                "name": "id",
                "type": "Int32",
                "precision": 10
            },
            {
                "name": "name",
                "type": "String"
            }
        ],
        "typeProperties": {
            "tableName": "[dbo].[table1]"
        }
    },
    "type": "Microsoft.DataFactory/factories/datasets"
}

希望这可以帮助。

Answer 2

从sql数据库移动到另一个数据库将类似于你已经完成的工作，但这次源将是一个sql表，就像你以前的接收器（或输出）一样。

因此，您应该做的是为新数据库创建一个新的链接服务，然后创建一个数据集作为输入（与您以前的输出格式相同，但更改链接的服务名称，以便它使用新的链接服务）。

最后，创建一个管道并正确配置输入和输出数据集。

希望这有帮助！