JDBC 查询与 JPA 查询性能

Question

从数据库读取数千条记录时，我遇到了一些与性能相关的问题。我注意到纯 JDBC 查询比 JPA 本机查询快得多。

这是查询

select ID, COL_A, COL_B, COL_C, COL_D, COL_E, COL_F from MY_SUPER_VIEW_V v 
where 1=1 
and v.ID in (:idList)
and v.DATE_FROM <= :date
and v.DATE_TILL >= :date;

此查询返回大约 38.000 条记录。

in idList 有超过 1000 条记录，因为我使用的是 Oracle DB，所以需要拆分为 n 个查询。

此外，我有一种方法可以将 Object[] 结果转换为我的List<Entity>.

为了理解性能问题，我分别创建了一个纯 JDBC 查询和一个 JPA Native 查询来比较结果。

以下是时间安排。

################ getScoresPureJDBCWithListIds ################
List of Ids retrieved. It took: 00:00:00.096 to execute query on DB using JDBC
It took: 00:00:01.180 to execute query on DB using JDBC query
Creating 24206 Scores records from DB result It took: 00:00:04.440
It took: 00:00:01.038 to execute query on DB using JDBC query
Creating 14445 Scores records from DB result It took: 00:00:04.307
################ getScoresJPANativeQueryWithListIds ################
It took: 00:06:09.450 to execute query on DB using JPA Native query
Creating 24206 Scores records from DB result It took: 00:00:00.009
It took: 00:04:04.879 to execute query on DB using JPA Native query
Creating 14445 Scores records from DB result It took: 00:00:00.007

使用 Hibernate 分析

################ USING FETCH_SIZE: 2000 ################
################ getSmartESGScoresPureJDBCWithListCsfLcIds ################
List of Securities CsfLcId retrieved. It took: 00:00:00.296 to execute query on DB using JDBC
It took: 00:00:11.940 to execute query on DB using JDBC query
Creating 24206 Smart Esg Scores records from DB result It took: 00:00:02.670
It took: 00:00:13.570 to execute query on DB using JDBC query
Creating 14445 Smart Esg Scores records from DB result It took: 00:00:02.553
################ getSmartESGScoresJDBCTemplateWithListCsfLcIds ################
List of Securities CsfLcId retrieved. It took: 00:00:00.087 to execute query on DB using JDBC
Creating 24206 Smart Esg Scores records from DB result It took: 00:00:04.063
Creating 14445 Smart Esg Scores records from DB result It took: 00:00:04.064
################ getSmartESGScoresJPANativeQueryAsESGenius with hint fetch size 2000 ################
2020-04-22 09:36:30.830  INFO 13262 --- [           main] i.StatisticalLoggingSessionEventListener : Session Metrics {
    1232369 nanoseconds spent acquiring 1 JDBC connections;
    0 nanoseconds spent releasing 0 JDBC connections;
    1448702 nanoseconds spent preparing 1 JDBC statements;
    3992364 nanoseconds spent executing 1 JDBC statements;
    0 nanoseconds spent executing 0 JDBC batches;
    0 nanoseconds spent performing 0 L2C puts;
    0 nanoseconds spent performing 0 L2C hits;
    0 nanoseconds spent performing 0 L2C misses;
    0 nanoseconds spent executing 0 flushes (flushing a total of 0 entities and 0 collections);
    0 nanoseconds spent executing 0 partial-flushes (flushing a total of 0 entities and 0 collections)
}
List of Securities CsfLcId retrieved. It took: 00:00:00.261 to execute query on DB using JDBC
2020-04-22 09:47:23.739  INFO 13262 --- [           main] i.StatisticalLoggingSessionEventListener : Session Metrics {
    73670 nanoseconds spent acquiring 1 JDBC connections;
    0 nanoseconds spent releasing 0 JDBC connections;
    805772 nanoseconds spent preparing 1 JDBC statements;
    651947762290 nanoseconds spent executing 1 JDBC statements; ==> 10 minutes
    0 nanoseconds spent executing 0 JDBC batches;
    0 nanoseconds spent performing 0 L2C puts;
    0 nanoseconds spent performing 0 L2C hits;
    0 nanoseconds spent performing 0 L2C misses;
    0 nanoseconds spent executing 0 flushes (flushing a total of 0 entities and 0 collections);
    0 nanoseconds spent executing 0 partial-flushes (flushing a total of 0 entities and 0 collections)
}
It took: 00:10:52.898 to execute query on DB using JPA Native query
Creating 24206 Smart Esg Scores records from DB result It took: 00:00:00.018
2020-04-22 09:56:00.792  INFO 13262 --- [           main] i.StatisticalLoggingSessionEventListener : Session Metrics {
    2758010 nanoseconds spent acquiring 1 JDBC connections;
    0 nanoseconds spent releasing 0 JDBC connections;
    3096653 nanoseconds spent preparing 1 JDBC statements;
    516148003151 nanoseconds spent executing 1 JDBC statements;
    0 nanoseconds spent executing 0 JDBC batches;
    0 nanoseconds spent performing 0 L2C puts;
    0 nanoseconds spent performing 0 L2C hits;
    0 nanoseconds spent performing 0 L2C misses;
    0 nanoseconds spent executing 0 flushes (flushing a total of 0 entities and 0 collections);
    0 nanoseconds spent executing 0 partial-flushes (flushing a total of 0 entities and 0 collections)
}
It took: 00:08:37.032 to execute query on DB using JPA Native query
Creating 14445 Smart Esg Scores records from DB result It took: 00:00:00.006

对于 JDBC 查询，我可以看到 1) 执行查询的速度非常快，但是 2) 处理循环中的每个 ResultSet 元素花费了大部分时间 00:09 秒 int total

另一方面对于 JPA Native 查询 1) 通过调用 query.getResultList() 方法执行查询需要大量时间 10:14 秒另一方面 2) 处理每个结果在这里非常快。分析表明，执行 1 条 JDBC 语句花费了大量时间。即使 FETCH_SIZE = 2000 也没有发生显着变化。

与纯 JDBC 相比，为什么 JPA Native 很慢？会是类型转换吗？就我而言，我指的是 varchar2 和数字。我期待与 JDBC 相同的结果。但从 8 秒到 10 分钟它很多。

我可以做些什么来改进 JPA Native 查询？

Answer 1

您似乎比较两个不同的查询，很可能导致数据库提出不同的查询计划。

有很多方法可以调查该问题，但我们无法使用这些方法，因为您没有提供最小的可重现示例。因此，我会建议一些选项供您自行调查：

• 为 Java 应用程序启用调试日志记录，包括 Hibernate 和 Oracle JDBC 驱动程序，如其文档中所述
• 观察延迟来自哪里，是数据库、网络还是 Java 应用程序？如果有疑问，请使用 Wireshark 检查连接两端的网络流量，或检查有问题的查询前后慢/重查询的 Oracle 数据库统计信息
• 如果问题是数据库速度慢，请确保您的查询参数具有与数据库索引匹配的类型
• 如果您确定网络和数据库不会导致问题，并且调试日志记录无法帮助您进一步尝试使用高级工具，例如带有 JVisualVM 的 cpu 分析器
• 如果您仍然遇到问题，则可能存在一些极端的内存问题，例如系统内存过少导致交换或非常频繁的完整垃圾收集，您可以从垃圾收集日志记录中看到这一点