在 PowerShell 中将大型 Blob 从 SQL Server 提取到文件需要很长时间

Question

我一直在研究一种自动将 blob 列提取到文件的方法，这个博客详细介绍了几种实现方法。

通过 BCP，可以非常快速地从我的数据库中提取较大的文件。我能够在 20 秒内提取 2 GB 的文件。这是我使用的示例命令行，基于博客中的示例：

BCP "SELECT PictureData FROM BLOB_Test.dbo.PicturesTest " QUERYOUT C:\BLOBTest\BlobOut\WC.jpg -T -f "C:\BLOBTest\FormatFile\BLOB.fmt" -S <ServerName>\<InstanceName>

顺便说一句，我必须学习如何应用格式文件来防止将前缀字符串插入到文件中。此格式文件必须采用 BCP 的旧格式，因为该格式文件的较新 XML 版本具有“PREFIX_LENGTH”的架构条目，该条目可防止出现 0 值。

我宁愿使用 PowerShell 来提取 blob，但以下基于 TechNet 文章的代码需要大约两天的时间来处理相同的 2 GB blob，而不是 BCP 的 20 秒。

## https://social.technet.microsoft.com/wiki/contents/articles/890.export-sql-server-blob-data-with-powershell.aspx
## Export of "larger" SQL Server blob to file with GetBytes-Stream

# Configuration data
   $Server     = ".\<Instance>";         # SQL Server Instance
   $Database   = "Blob_Test";            # Name of database
   $Dest       = "C:\BLOBTest\BLOBOut\"; # Path to export to
   $bufferSize = 8192;                   # Stream buffer size in bytes

# Select-Statement for name & blob with filter
   $Sql = "Select 
              [PictureName],
              [PictureData]
           From 
              dbo.PicturesTest";

# Open ADO.NET Connection
   $con = New-Object Data.SqlClient.SqlConnection;
   $con.ConnectionString = "Data Source=$Server;" +
                           "Integrated Security=True;" +
                           "Initial Catalog=$Database";
   $con.Open();

# New Command and Reader
   $cmd = New-Object Data.SqlClient.SqlCommand $Sql, $con;
   $rd  = $cmd.ExecuteReader();

# Create a byte array for the stream
   $out = [array]::CreateInstance('Byte', $bufferSize)

# Loop through records
   While ($rd.Read()) {
      Write-Output ("Exporting: {0}" -f $rd.GetString(0));
      
      # New BinaryWriter
         $fs = New-Object System.IO.FileStream ($Dest + $rd.GetString(0)), Create, Write;
         $bw = New-Object System.IO.BinaryWriter $fs;
      
         $start = 0;

      # Read first byte stream
         $received = $rd.GetBytes(1, $start, $out, 0, $bufferSize - 1);
      
      While ($received -gt 0) {
         $bw.Write($out, 0, $received);
         $bw.Flush();
         $start += $received;
         
         # Read next byte stream
            $received = $rd.GetBytes(1, $start, $out, 0, $bufferSize - 1);
      }
      
      $bw.Close();
      $fs.Close();
   }

# Closing & disposing all objects
   $fs.Dispose();
   $rd.Close();
   $cmd.Dispose();
   $con.Close();

Write-Output ("Finished");

它最终完成了，但我不知道为什么脚本需要这么长时间才能完成。

有人知道为什么 PowerShell 脚本被阉割吗？

Answer 1

您根本不需要 BinaryWriter。该类“仅”意味着以 .NET 特定格式编写基本类型，如整数、双精度数、字符串等。很少用。 如果您想将字节写入文件，您只需使用

Stream.Write

: $fs.Write $received

可以消除几乎所有代码的更好想法是使用 

DbDataReader.GetStream

而不是 GetBytes 将 BLOB 作为流读取。之后，您可以使用

Stream.CopyTo

将流的内容写入另一个流：

$dbFs=$rd.GetStream(1); 
$dbFs.CopyTo($fs);

Answer 2

$server = ".";
$database = "YourDatab";
$query = "SELECT FileContent,FileName FROM dbo.FileUploads";
$dirPath = "C:\Data\"
 
$connection=new-object System.Data.SqlClient.SQLConnection
$connection.ConnectionString="Server={0};Database={1};Integrated Security=True" -f $server,$database
$command=new-object system.Data.SqlClient.SqlCommand($query,$connection)
$command.CommandTimeout=120
$connection.Open()
$reader = $command.ExecuteReader()
while ($reader.Read())
{
    $sqlBytes = $reader.GetSqlBytes(0)
    $filepath = "$dirPath{0}" -f $reader.GetValue(1)
    $buffer = new-object byte[] -ArgumentList $reader.GetBytes(0,0,$null,0,$sqlBytes.Length)
    $reader.GetBytes(0,0,$buffer,0,$buffer.Length)
    $fs = new-object System.IO.FileStream($filePath,[System.IO.FileMode]'Create',[System.IO.FileAccess]'Write')
    $fs.Write($buffer, 0, $buffer.Length)
    $fs.Close()
}
$reader.Close()
$connection.Close()

来源：

https://www.sqlservercentral.com/blogs/t-sql-tuesday-006-blobs-filestream-and-powershell

Answer 3

# Loop through records
   While ($rd.Read()) {
      Write-Output ("Exporting: {0}" -f $rd.GetString(0));      

      # New file object for stream
         $fs = New-Object System.IO.FileStream ($Dest + $rd.GetString(0)), Create, Write

      # Stream [PictureData] blob to file
         $rd.GetStream(1).CopyTo($fs)

      $fs.Close()
   }