优化MERGE语句的SORT MERGE连接

Question

考虑将更改应用于聚合表的问题.必须更新存在的行,同时必须插入新行.我的方法如下:

• 在临时表中插入所有更改(一次100K)
• 将临时表合并到主表中(最终达到100万行)

SQL(带有SORT MERGE提示)如下所示(没什么特别的):

merge /*+ USE_MERGE(t s) */ 
into F_SCREEN_INSTANCE t
using F_SCREEN_INSTANCE_BUF s 
    on (s.DAY_ID = t.DAY_ID and s.PARTIAL_ID = t.PARTIAL_ID)
when matched then update set 
    t.ACTIVE_TIME_SUM = t.ACTIVE_TIME_SUM + s.ACTIVE_TIME_SUM,
    t.IDLE_TIME_SUM = t.IDLE_TIME_SUM + s.IDLE_TIME_SUM
when not matched then insert values (
    s.DAY_ID, s.PARTIAL_ID, s.ID, s.AGENT_USER_ID, s.COMPUTER_ID, s.RAW_APPLICATION_ID, s.APP_USER_ID, s.APPLICATION_ID, s.USER_ID, s.RAW_MODULE_ID, s.MODULE_ID, s.START_TIME, s.RAW_SCREEN_NAME, s.SCREEN_ID, s.SCREEN_TYPE, s.ACTIVE_TIME_SUM, s.IDLE_TIME_SUM)

该F_SCREEN_INSTANCE表具有(DAY_ID, PARTIAL_ID)主键,也是IOT(索引组织表).这使其成为合并连接的理想候选者:行按查找键进行物理排序.

到现在为止还挺好.我已经开始了一个基准测试,初始时间看起来不错,合并时间为10秒.但大约一个小时后,合并大约需要4分钟,因为tempdb使用率很高(每个合并4GB).下面的查询计划显示F_SCREEN_INSTANCE在合并之前重新排序,即使表已经理想地排序.当然,随着表的增长,将需要更多的tempdb,整个方法就会崩溃.

好的,为什么要重新排序表呢？它变成了合并连接实现的限制:第二个表总是排序的.

如果存在索引,则数据库可以避免对第一个数据集进行排序.但是,无论索引如何,数据库始终会对第二个数据集进行排序.

O ... K,那么我可以将主表放在第一位,将缓冲放在第二位吗？不,那也不可能.无论我如何列出USE_MERGE提示中的表,源表始终是第一个.

最后,这是我的问题:我错过了什么吗？是否有可能使这种SORT MERGE方法有效？

以下是一些解决您可能会问的问题的详细信息:

• 什么是Oracle版本？12C.
• 你试过HASH JOIN吗？是的,正如预期的那样糟糕.需要扫描主表以构建哈希表.它随着F_SCREEN_INSTANCE的增长而无法扩展.
• 你试过LOOP JOIN吗？是的,这也很糟糕.考虑到缓冲表的大小,在F_SCREEN_INSTANCE中进行100K查找会花费不合理的时间.合并大约需要3分钟.
• 总而言之,MERGE JOIN在概念上是最佳访问策略,但Oracle实现似乎因重新排序目标表而严重削弱.

Answer 1

无论提示如何,排序合并外连接总是将外连接表放在第二位.添加额外的内连接允许控制连接顺序,然后可以使用ROWID再次连接到大表.希望两个好的连接比一个坏的连接更好.

假设

此答案假定排序合并连接是最快的连接,并且手册是正确的,第二个数据集始终排序.如果没有关于数据的更多信息,就很难测试这些假设.

示例模式

下面是一些类似的表,使用虚假统计信息使优化器认为它们有500M行和100K行.

create table F_SCREEN_INSTANCE(DAY_ID number, PARTIAL_ID number, ID number, AGENT_USER_ID number,COMPUTER_ID number, RAW_APPLICATION_ID number, APP_USER_ID number, APPLICATION_ID number, USER_ID number, RAW_MODULE_ID number,MODULE_ID number, START_TIME date, RAW_SCREEN_NAME varchar2(100), SCREEN_ID number, SCREEN_TYPE number, ACTIVE_TIME_SUM number, IDLE_TIME_SUM number,
    constraint f_screen_instance_pk primary key (day_id, partial_id)
) organization index;

create table F_SCREEN_INSTANCE_BUF(DAY_ID number, PARTIAL_ID number, ID number, AGENT_USER_ID number,COMPUTER_ID number, RAW_APPLICATION_ID number, APP_USER_ID number,APPLICATION_ID number, USER_ID number, RAW_MODULE_ID number, MODULE_ID number, START_TIME date, RAW_SCREEN_NAME varchar2(100), SCREEN_ID number, SCREEN_TYPE number, ACTIVE_TIME_SUM number, IDLE_TIME_SUM number,
    constraint f_screen_instance_buf_pk primary key (day_id, partial_id)
);

begin
    dbms_stats.set_table_stats(user, 'F_SCREEN_INSTANCE', numrows => 500000000);
    dbms_stats.set_table_stats(user, 'F_SCREEN_INSTANCE_BUF', numrows => 100000);
end;
/

问题

使用内部联接时,可以使用LEADING提示实现所需的连接和连接顺序.较小的表F_SCREEN_INSTANCE_BUF是第二个表.

explain plan for
select /*+ use_merge(t s) leading(t s) */ *
from f_screen_instance_buf s
join f_screen_instance t
    on (s.DAY_ID = t.DAY_ID and s.PARTIAL_ID = t.PARTIAL_ID);

select * from table(dbms_xplan.display(format => '-predicate'));

Plan hash value: 563239985

-----------------------------------------------------------------------------------------------------
| Id  | Operation           | Name                  | Rows  | Bytes |TempSpc| Cost (%CPU)| Time     |
-----------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT    |                       |   100K|    19M|       |  6898  (66)| 00:00:01 |
|   1 |  MERGE JOIN         |                       |   100K|    19M|       |  6898  (66)| 00:00:01 |
|   2 |   INDEX FULL SCAN   | F_SCREEN_INSTANCE_PK  |   500M|    46G|       |  4504 (100)| 00:00:01 |
|   3 |   SORT JOIN         |                       |   100K|  9765K|    26M|  2393   (1)| 00:00:01 |
|   4 |    TABLE ACCESS FULL| F_SCREEN_INSTANCE_BUF |   100K|  9765K|       |    34   (6)| 00:00:01 |
-----------------------------------------------------------------------------------------------------

更改为左连接时,LEADING提示不起作用.

explain plan for
select /*+ use_merge(t s) leading(t s) */ *
from f_screen_instance_buf s
left join f_screen_instance t
    on (s.DAY_ID = t.DAY_ID and s.PARTIAL_ID = t.PARTIAL_ID);

select * from table(dbms_xplan.display(format => '-predicate'));

Plan hash value: 1472690071

-----------------------------------------------------------------------------------------------------------------
| Id  | Operation                    | Name                     | Rows  | Bytes |TempSpc| Cost (%CPU)| Time     |
-----------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT             |                          |   100K|    19M|       |    16M  (1)| 00:10:34 |
|   1 |  MERGE JOIN OUTER            |                          |   100K|    19M|       |    16M  (1)| 00:10:34 |
|   2 |   TABLE ACCESS BY INDEX ROWID| F_SCREEN_INSTANCE_BUF    |   100K|  9765K|       |   826   (0)| 00:00:01 |
|   3 |    INDEX FULL SCAN           | F_SCREEN_INSTANCE_BUF_PK |   100K|       |       |    26   (0)| 00:00:01 |
|   4 |   SORT JOIN                  |                          |   500M|    46G|   131G|    16M  (1)| 00:10:34 |
|   5 |    INDEX FAST FULL SCAN      | F_SCREEN_INSTANCE_PK     |   500M|    46G|       |  2703 (100)| 00:00:01 |
-----------------------------------------------------------------------------------------------------------------

据我所知,这个限制没有记录.我尝试使用+outline设置DBMS_XPLAN查看完整的提示,然后更改它们.但我没有做任何事情可以使LEFT JOIN版本的连接顺序更改.也许其他人可以让这个工作.

select * from table(dbms_xplan.display(format => '-predicate +outline'));

... 
Outline Data
-------------

  /*+
      BEGIN_OUTLINE_DATA
      USE_MERGE(@"SEL$0E991E55" "T"@"SEL$1")
      LEADING(@"SEL$0E991E55" "S"@"SEL$1" "T"@"SEL$1")
      INDEX_FFS(@"SEL$0E991E55" "T"@"SEL$1" ("F_SCREEN_INSTANCE"."DAY_ID" "F_SCREEN_INSTANCE"."PARTIAL_ID"))
      INDEX(@"SEL$0E991E55" "S"@"SEL$1" ("F_SCREEN_INSTANCE_BUF"."DAY_ID" 
              "F_SCREEN_INSTANCE_BUF"."PARTIAL_ID"))
      OUTLINE(@"SEL$9EC647DD")
      OUTLINE(@"SEL$2")
      MERGE(@"SEL$9EC647DD")
      OUTLINE_LEAF(@"SEL$0E991E55")
      ALL_ROWS
      DB_VERSION('12.1.0.1')
      OPTIMIZER_FEATURES_ENABLE('12.1.0.1')
      IGNORE_OPTIM_EMBEDDED_HINTS
      END_OUTLINE_DATA
  */

可能解决方案

--#3: Join the large table to the smaller result set.  This uses the largest table twice,
--but the plan can use the ROWID for a very quick join.
explain plan for
merge into F_SCREEN_INSTANCE t
using
(
    --#2: Now get the missing rows with an outer join.  Since the _BUF table is
    --small I assume it does not make a big difference exactly how it it joind
    --to the 100K result set.
    --The hints NO_MERGE and NO_PUSH_PRED are required to keep the INNER_JOIN
    --inline view intact.
    select /*+ no_merge(inner_join) no_push_pred(inner_join) */ inner_join.*
    from f_screen_instance_buf s
    left join
    (
        --#1: Get 100K rows efficiently with an inner join.
        --Note that the ROWID is retrieved here.
        select /*+ use_merge(t s) leading(t s) */ s.*, s.rowid s_rowid
        from f_screen_instance_buf s
        join f_screen_instance t
            on (s.DAY_ID = t.DAY_ID and s.PARTIAL_ID = t.PARTIAL_ID)
    ) inner_join
        on (s.DAY_ID = inner_join.DAY_ID and s.PARTIAL_ID = inner_join.PARTIAL_ID)
) s
    on (s.s_rowid = t.rowid)
when matched then update set 
    t.ACTIVE_TIME_SUM = t.ACTIVE_TIME_SUM + s.ACTIVE_TIME_SUM,
    t.IDLE_TIME_SUM = t.IDLE_TIME_SUM + s.IDLE_TIME_SUM
when not matched then insert values (
    s.DAY_ID, s.PARTIAL_ID, s.ID, s.AGENT_USER_ID, s.COMPUTER_ID, s.RAW_APPLICATION_ID, s.APP_USER_ID, s.APPLICATION_ID, s.USER_ID, s.RAW_MODULE_ID, s.MODULE_ID, s.START_TIME, s.RAW_SCREEN_NAME, s.SCREEN_ID, s.SCREEN_TYPE, s.ACTIVE_TIME_SUM, s.IDLE_TIME_SUM);

它并不漂亮,但至少它会在排序合并连接中首先生成一个包含大表的计划.

select * from table(dbms_xplan.display);

Plan hash value: 1086560566

-------------------------------------------------------------------------------------------------------------
| Id  | Operation                | Name                     | Rows  | Bytes |TempSpc| Cost (%CPU)| Time     |
-------------------------------------------------------------------------------------------------------------
|   0 | MERGE STATEMENT          |                          |   500G|   173T|       |  5355K (43)| 00:03:30 |
|   1 |  MERGE                   | F_SCREEN_INSTANCE        |       |       |       |            |          |
|   2 |   VIEW                   |                          |       |       |       |            |          |
|*  3 |    HASH JOIN OUTER       |                          |   500G|   179T|    29M|  5355K (43)| 00:03:30 |
|*  4 |     HASH JOIN OUTER      |                          |   100K|    28M|  3712K|  8663  (53)| 00:00:01 |
|   5 |      INDEX FAST FULL SCAN| F_SCREEN_INSTANCE_BUF_PK |   100K|  2539K|       |     9   (0)| 00:00:01 |
|   6 |      VIEW                |                          |   100K|    25M|       |  6898  (66)| 00:00:01 |
|   7 |       MERGE JOIN         |                          |   100K|    12M|       |  6898  (66)| 00:00:01 |
|   8 |        INDEX FULL SCAN   | F_SCREEN_INSTANCE_PK     |   500M|    12G|       |  4504 (100)| 00:00:01 |
|*  9 |        SORT JOIN         |                          |   100K|  9765K|    26M|  2393   (1)| 00:00:01 |
|  10 |         TABLE ACCESS FULL| F_SCREEN_INSTANCE_BUF    |   100K|  9765K|       |    34   (6)| 00:00:01 |
|  11 |     INDEX FAST FULL SCAN | F_SCREEN_INSTANCE_PK     |   500M|    46G|       |  2703 (100)| 00:00:01 |
-------------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   3 - access("INNER_JOIN"."S_ROWID"=("T".ROWID(+)))
   4 - access("S"."PARTIAL_ID"="INNER_JOIN"."PARTIAL_ID"(+) AND 
              "S"."DAY_ID"="INNER_JOIN"."DAY_ID"(+))
   9 - access("S"."DAY_ID"="T"."DAY_ID" AND "S"."PARTIAL_ID"="T"."PARTIAL_ID")
       filter("S"."PARTIAL_ID"="T"."PARTIAL_ID" AND "S"."DAY_ID"="T"."DAY_ID")