通过java spring boot上传Bigquery 400错误csv

Question

我正在尝试使用 spring-boot java 将 csv 文件上传到 bigquery 我收到 400 条无效数据

public void uploadToBigQuery(File csvFile, String table, boolean overwrite, List<Pair<String, String>> columns) {
        log.info("Upload CSV file [{}] to BigQuery table [{}/{}]", csvFile.getPath(), dataSetName, table);

        try {
            // Fields
            List<Field> fields = columns.stream()
                    .map(c -> Field.of(c.getFirst(), StandardSQLTypeName.valueOf(c.getSecond())))
                    .collect(Collectors.toList());

            TableId tableId = TableId.of(dataSetName, table);
            WriteChannelConfiguration writeChannelConfiguration =
                    WriteChannelConfiguration
                            .newBuilder(tableId)
                            .setFormatOptions(FormatOptions.csv())
                            .setCreateDisposition(JobInfo.CreateDisposition.CREATE_IF_NEEDED)
                            .setWriteDisposition(overwrite ? JobInfo.WriteDisposition.WRITE_TRUNCATE : JobInfo.WriteDisposition.WRITE_APPEND)
                            .setSchema(Schema.of(fields))
                            .build();

            // The location must be specified; other fields can be auto-detected.
            JobId jobId = JobId.newBuilder().setLocation(DATASET_LOCATION).build();
            TableDataWriteChannel writer = bigQuery.writer(jobId, writeChannelConfiguration);

            // Write data to writer
            try (OutputStream stream = Channels.newOutputStream(writer)) {
                Files.copy(csvFile.toPath(), stream);
            }

            // Get load job
            Job job = writer.getJob();
            job = job.waitFor();
            System.err.println("BigQuery job error: " + job.getStatus().getError().toString());
            System.err.println("Full error: " + job.getStatus());
            JobStatistics.LoadStatistics stats = job.getStatistics();
            log.info("Wrote {} records to BigQuery table [{}/{}]", stats.getOutputRows(), dataSetName, table);
        } catch (IOException | InterruptedException | BigQueryException ex) {
            log.error("Error occured during writing of file to BigQuery", ex);
        }
    }

但出现以下错误

2024-07-04T23:28:06.389+05:30  INFO 32704 --- [           main] c.google.api.client.http.HttpTransport   : {
  "error": {
    "code": 400,
    "message": "Error: 3848323",
    "errors": [
      {
        "message": "Error: 3848323",
        "domain": "global",
        "reason": "invalid"
      }
    ],
    "status": "INVALID_ARGUMENT"
  }
}

为什么会出现此错误以及如何排除故障

Answer 1

有同样的问题。对我来说，改变

JobId.newBuilder().setLocation("us") // copied from a code sample

到

JobId.newBuilder().setLocation("eu") // where my project is actually located

修正了错误。