如何使用 Symfony 导出大型 CSV 文件的技术

Question

我的任务是将数据库中的 100,000 行记录导出为 csv 格式。

执行此操作的最佳方法是什么？我完全不知道要搜索和研究什么。

我想像其他PART格式的网站一样在下载尚未完成时进行块下载，同时不要让我的服务器耗尽。

我该怎么做？

谢谢！

Answer 1

您可以将 Symfony HttpFoundation 组件中的 StreamedResponse 与 Doctrine iterate 结合使用。

类似这样的：

$response = new StreamedResponse(function () {
    $data = $this->myQuery()->iterate();
    $csv = fopen('php://output', 'w+');
    while (false !== ($line = $data->next())) {
        fputcsv($csv, [$line[0]->column1], ';');
    }
    fclose($csv);
});
$response->headers->set('Content-Type', 'text/csv; charset=utf-8');
$response->headers->set('Content-Disposition', 'attachment; filename="file.csv"');

Answer 2

基于@G1.3的回答，这里是Doctrine 2.11+（现在最新版本，2022年5月25日）和Symfony 5.4的更新代码：

class CsvExportController extends AbstractController
{
    public function downloadCsv(
        EntityManagerInterface $entityManager
    ): StreamedResponse {
        $response = new StreamedResponse(function () use ($entityManager) {
            $csv = fopen('php://output', 'w+');
            fputcsv($csv, [
                'your',
                'csv_headers',
                'go_here'
            ]);
            $query = $entityManager->createQuery('SELECT t FROM Your\Entity\Class t');
            /**
             * @var \Your\Entity\Class $row
             */
            foreach ($query->toIterable() as $row) {
                fputcsv($csv, [
                    $row->getYourDataField01,
                    $row->getYourDataField02,
                    $row->getYourDataField03
                ]);
                // detach from Doctrine, so that it can be Garbage-Collected immediately
                $entityManager->detach($row);
            }
            fclose($csv);
        });
        
    $response->headers->set('Content-Type', 'text/csv; charset=utf-8');
    $response->headers->set('Content-Disposition', 'attachment; filename="file.csv"');
    $response->headers->set('Cache-Control', 'no-store');

    return $response;
    }
}

Answer 3

你可以这样做：

控制器功能：

private function downloadCSV(QueryBuilder $query): Response
    {
        $csv = $this->renderView('templates/csv/template.csv.twig', [
            'data' => $query->getQuery()->getResult(),
        ]);

        // crafting response
        $response = new Response($csv);

        $disposition = $response->headers->makeDisposition(
            ResponseHeaderBag::DISPOSITION_ATTACHMENT,
            sprintf('file_%s.csv', time()));
        $response->headers->set('Content-Type', 'text/csv');
        $response->headers->set('Content-Disposition', $disposition);

        return $response;
    }

在模板中渲染 CSV：

{% set header = [
    'Internal Id',
    'title',
    'url'
] %}
{{ str_putcsv(header) }}
{% for i in data %}
{% spaceless %}
    {% set row = [
        i.id,
        i.title,
        absolute_url(path('route_to_post', { data: i.id }))
    ] %}
{% endspaceless %}
{{ str_putcsv(row) }}
{% endfor %}

Answer 4

1. 作曲家需要 symfony/http-foundation

<?php

include_once "./vendor/autoload.php";

$header = [
    'Content-Encoding'    => 'UTF-8',
    'Content-Type'        => 'text/csv;charset=UTF-8',
    'Content-Disposition' => "attachment;filename=\"mycsv.csv\"",
];

$response = new \Symfony\Component\HttpFoundation\StreamedResponse(function() {
    $handle = fopen('php://output', 'w');

    // here write your db record
    while (true) {
        fputcsv($handle, ['a', 'b', 'c']);
    }

    fclose($handle);
}, 200, $header);

$response->send();
exit();