Sql Server删除和合并性能

Question

我的表中包含一些买入/卖出数据,其中包含大约8M的记录:

CREATE TABLE [dbo].[Transactions](
[id] [int] IDENTITY(1,1) NOT NULL,
[itemId] [bigint] NOT NULL,
[dt] [datetime] NOT NULL,
[count] [int] NOT NULL,
[price] [float] NOT NULL,
[platform] [char](1) NOT NULL
) ON [PRIMARY]

每个X分钟我的程序为每个itemId获取新的事务,我需要更新它.我的第一个解决方案是两步DELETE + INSERT:

delete from Transactions where platform=@platform and itemid=@itemid
insert into Transactions (platform,itemid,dt,count,price) values (@platform,@itemid,@dt,@count,@price)
[...]
insert into Transactions (platform,itemid,dt,count,price) values (@platform,@itemid,@dt,@count,@price)

问题是,这个DELETE语句平均需要5秒.这太长了.

我找到的第二个解决方案是使用MERGE.我创建了这样的存储过程,wchich采用表值参数:

CREATE PROCEDURE [dbo].[sp_updateTransactions]
@Table dbo.tp_Transactions readonly,
@itemId bigint,
@platform char(1)
AS
BEGIN
MERGE Transactions AS TARGET
USING @Table AS SOURCE  
ON (    
TARGET.[itemId] = SOURCE.[itemId] AND
TARGET.[platform] = SOURCE.[platform] AND 
TARGET.[dt] = SOURCE.[dt] AND 
TARGET.[count] = SOURCE.[count] AND
TARGET.[price] = SOURCE.[price] ) 


WHEN NOT MATCHED BY TARGET THEN 
INSERT VALUES (SOURCE.[itemId], 
                SOURCE.[dt],
                SOURCE.[count],
                SOURCE.[price],
                SOURCE.[platform])

WHEN NOT MATCHED BY SOURCE AND TARGET.[itemId] = @itemId AND TARGET.[platform] = @platform THEN 
DELETE;

END

对于具有70k记录的表,此过程大约需要7秒.所以8M可能需要几分钟.瓶颈是"当不匹配"时 - 当我评论此行时,此过程平均运行0,01秒.

所以问题是:如何提高删除语句的性能？

需要删除才能确保该表不包含已在应用程序中删除的事务.但真正的情况是它很少发生,在10000次交易更新中,删除记录的真正需求小于1.

我的理论解决方法是创建额外的列,如"transactionDeleted bit"并使用UPDATE而不是DELETE,然后每隔X分钟或几小时通过批处理作业清理表并执行

delete from transactions where transactionDeleted=1

它应该更快,但我需要更新应用程序的其他部分中的所有SELECT语句,仅使用transactionDeleted = 0记录,因此它也可能会影响应用程序性能.

你知道更好的解决方案吗？

更新:当前索引:

CREATE NONCLUSTERED INDEX [IX1] ON [dbo].[Transactions] 
(
[platform] ASC,
[ItemId] ASC
) WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, SORT_IN_TEMPDB = OFF,   IGNORE_DUP_KEY = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON, FILLFACTOR = 50) ON [PRIMARY]


CONSTRAINT [IX2] UNIQUE NONCLUSTERED 
(
[ItemId] DESC,
[count] ASC,
[dt] DESC,
[platform] ASC,
[price] ASC
) WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON) ON [PRIMARY]

Answer 1

好的，这也是另一种方法。对于类似的问题（大扫描，如果未按源匹配，则删除），我将合并执行时间从806ms减少到了6ms！

上述问题的一个问题是“未按源匹配时”子句正在扫描整个TARGET表。

并不是很明显，但是Microsoft允许在合并之前先过滤TARGET表（通过使用CTE）。因此，在我的情况下，TARGET行从250K减少到少于10行。巨大差距。

假设上述问题适用于@itemid和@platform过滤的TARGET，则MERGE代码将如下所示。索引上面的更改也将有助于此逻辑。

WITH Transactions_CTE (itemId
                        ,dt
                        ,count
                        ,price
                        ,platform
                        )
AS
-- Define the CTE query that will reduce the size of the TARGET table.  
(  
    SELECT itemId
        ,dt
        ,count
        ,price
        ,platform
    FROM Transactions  
    WHERE itemId = @itemId
      AND platform = @platform  
)  
MERGE Transactions_CTE AS TARGET
USING @Table AS SOURCE
    ON (
        TARGET.[itemId] = SOURCE.[itemId]
        AND TARGET.[platform] = SOURCE.[platform]
        AND TARGET.[dt] = SOURCE.[dt]
        AND TARGET.[count] = SOURCE.[count]
        AND TARGET.[price] = SOURCE.[price]
        )
WHEN NOT MATCHED BY TARGET  THEN
        INSERT
        VALUES (
            SOURCE.[itemId]
            ,SOURCE.[dt]
            ,SOURCE.[count]
            ,SOURCE.[price]
            ,SOURCE.[platform]
            )
WHEN NOT MATCHED BY SOURCE THEN
        DELETE;

Answer 2

请参阅此/sf/ask/257959901/....

更新的成本与删除的成本相同吗？不。更新将是一个更轻松的操作，特别是如果您在 PK 上有一个索引（呃，那是一个 guid，而不是一个 int）。关键是对位字段的更新要便宜得多。（批量）删除将强制重新调整数据。

根据这些信息，您使用位字段的想法是非常有效的。