Presto JDBC大查询非常慢

Question

问题

1. 使用JDBC驱动程序从Presto查询中遍历大ResultSet非常耗时。每个批处理调用似乎几乎需要60s，这是可疑的（可能是由于超时引起的？）。
2. 由于某种原因，最初的executeQuery方法几​​乎花费了45秒，这是可疑的。

查询

我正在使用Java JDBC Presto驱动程序对Presto执行简单的查询，本质上看起来像这样：

SELECT stringA, stringB 
FROM {table} 
LIMIT 500000

stringA和stringB很小-分别约为10个字符。使用Teradata驱动程序在DbVisualizer中运行时，我的查询在10秒内完成。但是，当我使用0.230 presto-jdbc驱动程序从Spring Java应用程序运行相同的查询时，似乎可以将结果分批返回（大约75,000），每批返回都需要一分钟以上的时间。

我已经阅读了有关Presto的targetResultSize查询参数的内容，但是我无法使用JDBC驱动程序/连接进行设置。我读过presto默认情况下一次只会返回1MB的数据？不知道这是否是我上述问题＃1的原因-很好地找出如何配置它。

Java代码

public List<Object> getResultSetUsingDriverManager(ChronoLocalDate chronoLocalDate) throws SQLException {
    long start = System.currentTimeMillis();

    Properties properties = new Properties();
    properties.setProperty("user", USERNAME);
    properties.setProperty("password", PASSWORD);
    properties.setProperty("SSL", "true");

    final Connection connection = DriverManager.getConnection(URL, properties);

    log.warn("Presto connection acquired in " + (System.currentTimeMillis() - start) + "ms");

    Statement statement = connection.createStatement();

    ResultSet resultSet = statement.executeQuery(query);

    log.warn("Presto query executed in " + (System.currentTimeMillis() - start) + "ms");

    List<Object> collection = new ArrayList<>();

    int counter = 0;
    long batchStart = System.currentTimeMillis();
    while (resultSet.next()) {
        counter++;
        if (counter % 1000 == 0) {
            log.warn("current count {} and took {}ms", counter, (System.currentTimeMillis() - batchStart));
            batchStart = System.currentTimeMillis();
        }
    }
    log.warn("Results extracted in " + (System.currentTimeMillis() - start));

    return collection;
}

输出

2020-01-08 17:34:31.704  WARN 29368 --- ...       : Presto connection acquired in 0ms
2020-01-08 17:35:16.705  WARN 29368 --- ...       : Presto query executed in 45003ms
2020-01-08 17:37:18.242  WARN 29368 --- ...       : current count 1000 and took 121537ms
2020-01-08 17:37:18.244  WARN 29368 --- ...       : current count 2000 and took 2ms
2020-01-08 17:37:18.245  WARN 29368 --- ...       : current count 3000 and took 1ms
...
2020-01-08 17:37:18.294  WARN 29368 --- ...       : current count 75000 and took 1ms
2020-01-08 17:38:18.857  WARN 29368 --- ...       : current count 76000 and took 60563ms
2020-01-08 17:38:18.858  WARN 29368 --- ...       : current count 77000 and took 1ms
...
2020-01-08 17:38:18.941  WARN 29368 --- ...       : current count 151000 and took 0ms
2020-01-08 17:39:19.241  WARN 29368 --- ...       : current count 152000 and took 60300ms
2020-01-08 17:39:19.242  WARN 29368 --- ...       : current count 153000 and took 1ms
...
2020-01-08 17:39:19.311  WARN 29368 --- ...       : current count 250000 and took 0ms
2020-01-08 17:39:19.311  WARN 29368 --- ...       : Results extracted in 287609

版本信息

• Java 11
• com.facebook.presto presto-jdbc 0.230
• Spring Boot 2.1.6.RELEASE
• Presto版本：302-e.3（Starburst版本）

Answer 1

最近几个月，我们修复了与Java 11上的客户端连接有关的问题。

请将JDBC驱动程序升级到327。

或在客户端降级到Java 8。