通过Java Webapp从查询中下载很大的结果

Question

我一直在思考一个问题，我想知道是否有人提出任何建议。我有一个Web应用程序，需要添加一个部分以向其报告。它将具有基于用户gui输入构造的t-sql查询，执行并允许用户通过浏览器将文件以csv的形式下载到其计算机上。就其本身而言，这没有一个过于复杂。我以前做过这些事情。但是，我以前没有解决过一个问题。结果可能最终成为价值数GB的数据。为了防止浏览器超载或速度过慢，我将如何与正常代码做些不同（在代码方面）？显然不能一次全部将其保存在内存中吗？如何防止这种情况发生？

Answer 1

您可以将csv数据直接写入传入请求的OutputStream：

package example;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import java.io.IOException;
import java.io.OutputStreamWriter;
import java.io.Writer;
import java.nio.charset.StandardCharsets;

public class MyServlet extends HttpServlet {

    private final CSVCreator csvCreator;

    private MyServlet(CSVCreator csvCreator) {
        this.csvCreator = csvCreator;
    }

    @Override
    protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
        // be nice to your consumer, tell them what they get
        resp.setHeader("Content-Type", "text/comma-separated-values; charset=UTF-8");

        try (Writer writer = new OutputStreamWriter(resp.getOutputStream(), StandardCharsets.UTF_8)) {
            this.csvCreator.createCSV("some params you need to create the correct csv", writer);
        }
    }
}

例如，您必须更改csv代才能写入Writer ...：>

package example;

import java.io.IOException;

public class CSVCreator {

    public void createCSV(String someParamsYouNeedToCreateTheCSV, Appendable appendable) throws IOException {
        appendable.append("Column1,Column2,Column3\n");
        appendable.append("Row1,Row1,Row1\n");
        appendable.append("Row2,Row2,Row2\n");
    }
}

例如，您也可以使用apache-commons-csv将csv写入给定的Appendable（请参阅：https://javadoc.io/doc/org.apache.commons/commons-csv/latest/org/apache/commons/csv/CSVPrinter.html]

package example;

import org.apache.commons.csv.CSVFormat;
import org.apache.commons.csv.CSVPrinter;

import java.io.IOException;

public class CSVCreator {

    public void createCSV(String someParamsYouNeedToCreateTheCSV, Appendable appendable) throws IOException {
        try (CSVPrinter printer = new CSVPrinter(appendable, CSVFormat.DEFAULT)) {
            printer.printRecord("Column1", "Column2", "Column3");
            printer.printRecord("Row1", "Row1", "Row1");
            printer.printRecord("Row2", "Row2", "Row2");
        }
    }
}
编辑：

您可以像这样从ResultSet创建CSV文件：

package example;

import org.apache.commons.csv.CSVFormat;
import org.apache.commons.csv.CSVPrinter;

import java.io.IOException;
import java.sql.ResultSet;
import java.sql.ResultSetMetaData;
import java.sql.SQLException;

public class ResultSetToCSV {

    public void createCSV(ResultSet rs, Appendable appendable) throws IOException, SQLException {
        final ResultSetMetaData metaData = rs.getMetaData();
        final int columnCount = metaData.getColumnCount();

        try (CSVPrinter printer = new CSVPrinter(appendable, CSVFormat.DEFAULT)) {
            printHeaders(metaData, columnCount, printer);
            printValues(rs, columnCount, printer);
        }
    }

    private void printHeaders(ResultSetMetaData metaData, int columnCount, CSVPrinter printer) throws IOException, SQLException {
        for (int i = 1; i <= columnCount; i++) {
            printer.print(metaData.getColumnLabel(i));
        }

        printer.println();
    }

    private void printValues(ResultSet rs, int columnCount, CSVPrinter printer) throws IOException, SQLException {
        while (rs.next()) {
            printValue(rs, columnCount, printer);
        }
    }

    private void printValue(ResultSet rs, int columnCount, CSVPrinter printer) throws IOException, SQLException {
        for (int i = 1; i <= columnCount; i++) {
            printer.print(rs.getString(i));
        }

        printer.println();
    }
}