涉及超过一百万行的 Top、Order By 和 Where 条件的性能问题

Question

涉及超过一百万行的 Top、Order By 和 Where 条件的性能问题

Moo*_*ons 3 sql-server stored-procedures sql-server-2016

我有一个查询，如下所示：

Declare @CompanyID as int = 10;

Select Top 50  UserID, AuditTypeID , [Action], ActionDate , [Description] , 
RecordID, S.ID as SiteUserID from AuditTrial A

inner Join SiteUser S on S.ID = A.UserID

where A.CompanyID = @CompanyID And A.AuditTypeID <> 1 
And 

UserID in 
(
    1,2,3 -- will be a subquery - will have upto 100+ rows.
)

Order by ID desc -- Latest rows

Run Code Online (Sandbox Code Playgroud)

这是一个稍大一点的查询的一部分，我暂时删除了一些额外的和条件，甚至这个剥离的查询运行得更慢。

关键是前 50 名、按 ID最新订购和带有用户 ID的where 条件。

该表包含超过 100 万行。用户 ID - 范围从 100 到 20000。

具有自动生成的主键
UserID、CompanyID 等索引

到目前为止我尝试过的

当 UserID where 条件的行数较少时，它运行得足够好 < 1 秒，但当用户数量增加时，它开始花费更多时间，即大于 10 秒。

有时使用 SqlBulkCopy 在此表中添加数千行，并且常规的正常插入很常见。

我已经替换了要加入的子查询 - 影响不大
首先将用户 ID 放在临时表中 - 同样没有太大影响
用 DataSchema 尝试了一些视图（谷歌搜索） - 仍然很慢

不是 DBA - 我不太了解聚集/非聚集索引。

任何帮助表示赞赏。

额外细节

除了已经创建索引的主键外，其他如下：

对于将在 where 列中使用的所有 in 列，如 UserID、CompanyID、PermissionID 等。我没有任何多列的非聚集索引

CREATE NONCLUSTERED INDEX [IX_AuditTrial_1] ON [dbo].[AuditTrial]
(
[UserID] ASC
)

Run Code Online (Sandbox Code Playgroud)

如果我在 ,In' 子句中用逗号分隔的 50 个值对 UserID 进行硬编码，则子查询会立即运行。

完整实时查询



declare  @SiteUserID as int = 6484,
 @CompanyID as int = 34;


DECLARE @ColleageUsers TABLE(
ID int
)

Insert into @ColleageUsers
Select SUB.SiteUserID from SiteUserBroker SUB (nolock)
inner join (Select Count(*) as TotalBrokers , SiteUserID from SiteUserBroker group by SiteUserID)Nested on Sub.SiteUserID = Nested.SiteUserID
where BrokerID in (
Select BrokerID from SiteUserBroker (nolock) where SiteUserID = @SiteUserID)
group by SUB.SiteUserID
Having Count(*) - MIN(TotalBrokers) >= 0


Select Top 50  UserID, AuditTypeID , [Action], ActionDate , [Description] , RecordID, ActionedBy =  S.FirstName + ' ' + S.LastName , S.ID as SiteUserID from AuditTrial (nolock) A
inner Join SiteUser (nolock) S on S.ID = A.UserID
where A.CompanyID = @CompanyID And A.AuditTypeID  1 -- 1 Means Audit Type Login
And
(
184 = (Select Distinct Top 1 PermissionID from PermissionGroup (nolock) where GroupID in 
(Select GroupID from SiteUserGroup (nolock) where SiteUserID = @SiteUserID) and PermissionID = 184)
OR
A.AuditTypeID  20   - Not equal to
)
And (A.PermissionID is null Or A.PermissionID in ( Select PermissionID from PermissionGroup (nolock) where GroupID in 
(Select GroupID from SiteUserGroup (nolock) where SiteUserID = @SiteUserID)))
And 
(
( 
UserID in 
(
  Select Id from @ColleageUsers


)
)
)

Order by A.id desc

查询计划

https://www.brentozar.com/pastetheplan/?id=SkQt9mGvr 计划链接

Answer 1

Ran*_*gen 9

以下是一些可以单独或全部尝试的方法。

选项（重新编译）

由于您使用的是在查询文本本身中声明的变量，因此您可以OPTION(RECOMPILE)在每次执行时重新创建查询计划。这样优化器在运行时“看到”变量中的值，你应该得到一个更适合这些值的查询计划。

禁用行目标

由于由于 TOP() 运算符而设置了行目标，因此您可以尝试使用跟踪标志禁用该行目标：4138。

连接运算符计划中的行目标示例：

在查询级别禁用行目标： OPTION(QUERYTRACEON 4138);

更多关于行目标的信息在这里

将这些与您的查询一起测试：

SELECT TOP 50  UserID, AuditTypeID , [Action], ActionDate , [Description] , RecordID, ActionedBy =  S.FirstName + ' ' + S.LastName , S.ID as SiteUserID 
FROM AuditTrial (nolock) A
INNER JOIN SiteUser (nolock) S on S.ID = A.UserID
WHERE 
A.CompanyID = @CompanyID 
AND A.AuditTypeID <> 1 -- 1 Means Audit Type Login
AND
(
184 = (Select Distinct Top 1 PermissionID from PermissionGroup (nolock) where GroupID in 
(Select GroupID from SiteUserGroup (nolock) where SiteUserID = @SiteUserID) and PermissionID = 184)
OR
A.AuditTypeID <> 20
)
AND (A.PermissionID 
        IS NULL 
     OR  A.PermissionID IN 
                ( SELECT PermissionID 
                    FROM PermissionGroup (nolock) 
                    WHERE GroupID in 
                                    (SELECT GroupID FROM SiteUserGroup (nolock) 
                                     WHERE SiteUserID = @SiteUserID)
                 )
    )
AND 
(
( 
UserID IN 
(
  SELECT Id FROM @ColleageUsers
)
)
)
ORDER BY A.id desc
OPTION(RECOMPILE,QUERYTRACEON 4138 );

Run Code Online (Sandbox Code Playgroud)

删除 OR 并使用 UNION ALL

其他确实需要您重写查询的东西是将OR's 与UNION ALL's分开。

通过这种方式，可以以处理更多数据为代价更有效地使用索引/运算符。这取决于使用的过滤器和索引。

您的查询示例：

SELECT TOP 50  UserID, AuditTypeID , [Action], ActionDate , [Description] , RecordID, ActionedBy,SiteUserID 

FROM
( 
SELECT UserID, AuditTypeID , [Action], ActionDate , [Description] , RecordID, ActionedBy =  S.FirstName + ' ' + S.LastName , S.ID as SiteUserID , A.id
FROM AuditTrial (nolock) A
INNER JOIN SiteUser (nolock) S on S.ID = A.UserID
WHERE 
A.CompanyID = @CompanyID 
AND A.AuditTypeID <> 1 -- 1 Means Audit Type Login
AND
(
184 = (Select Distinct Top 1 PermissionID from PermissionGroup (nolock) where GroupID in 
(Select GroupID from SiteUserGroup (nolock) where SiteUserID = @SiteUserID) and PermissionID = 184)
OR
A.AuditTypeID <> 20
)
AND (A.PermissionID 
        IS NULL 
    )
AND 
(
( 
UserID IN 
(
  SELECT Id FROM @ColleageUsers
)
)
)
UNION ALL
SELECT TOP 50  UserID, AuditTypeID , [Action], ActionDate , [Description] , RecordID, ActionedBy =  S.FirstName + ' ' + S.LastName , S.ID as SiteUserID ,A.id
FROM AuditTrial (nolock) A
INNER JOIN SiteUser (nolock) S on S.ID = A.UserID
WHERE 
A.CompanyID = @CompanyID 
AND A.AuditTypeID <> 1 -- 1 Means Audit Type Login
AND
(
184 = (Select Distinct Top 1 PermissionID from PermissionGroup (nolock) where GroupID in 
(Select GroupID from SiteUserGroup (nolock) where SiteUserID = @SiteUserID) and PermissionID = 184)
OR
A.AuditTypeID <> 20
)
 AND  A.PermissionID IN 
                ( SELECT PermissionID 
                    FROM PermissionGroup (nolock) 
                    WHERE GroupID in 
                                    (SELECT GroupID FROM SiteUserGroup (nolock) 
                                     WHERE SiteUserID = @SiteUserID)
                 )

AND 
(
( 
UserID IN 
(
  SELECT Id FROM @ColleageUsers
)
)
)) AS A;

ORDER BY A.id desc
OPTION(RECOMPILE,QUERYTRACEON 4138 );

Run Code Online (Sandbox Code Playgroud)

表变量/临时表

我会尝试的另一件事是将表变量更改为临时表

CREATE TABLE #ColleageUsers(
ID int
)

Run Code Online (Sandbox Code Playgroud)

获取临时对象的统计信息，因为表变量没有统计信息，但临时表有。

使用临时表将查询拆分为多个部分

尝试获得不同执行计划的另一件事是预先做一些工作，并将这些结果存储到临时表中。当接触到很多表时，就很难得到正确的估计，拆分工作可以更容易地计算每个部分的估计。

您的查询示例：

declare  @SiteUserID as int = 6484,
 @CompanyID as int = 34;

SELECT PermissionID 
INTO #TEMP
FROM PermissionGroup (nolock) 
WHERE GroupID in 
(SELECT GroupID FROM SiteUserGroup (nolock) 
 WHERE SiteUserID = @SiteUserID)

DECLARE @PermissionID INT
SELECT DISTINCT Top 1 @PermissionID=PermissionID 
FROM PermissionGroup (nolock) where GroupID in 
(Select GroupID from SiteUserGroup (nolock) 
where SiteUserID = @SiteUserID) and PermissionID = 184


SELECT TOP 50  UserID, AuditTypeID , [Action], ActionDate , [Description] , RecordID, ActionedBy,SiteUserID 

FROM
( 
SELECT UserID, AuditTypeID , [Action], ActionDate , [Description] , RecordID, ActionedBy =  S.FirstName + ' ' + S.LastName , S.ID as SiteUserID , A.id
FROM AuditTrial (nolock) A
INNER JOIN SiteUser (nolock) S on S.ID = A.UserID
WHERE 
A.CompanyID = @CompanyID 
AND A.AuditTypeID <> 1 -- 1 Means Audit Type Login
AND
(
184 = (@PermissionID)
OR
A.AuditTypeID <> 20
)
AND (A.PermissionID 
        IS NULL 
    )
AND 
(
( 
UserID IN 
(
  SELECT Id FROM @ColleageUsers
)
)
)
UNION ALL
SELECT TOP 50  UserID, AuditTypeID , [Action], ActionDate , [Description] , RecordID, ActionedBy =  S.FirstName + ' ' + S.LastName , S.ID as SiteUserID ,A.id
FROM AuditTrial (nolock) A
INNER JOIN SiteUser (nolock) S on S.ID = A.UserID
WHERE 
A.CompanyID = @CompanyID 
AND A.AuditTypeID <> 1 -- 1 Means Audit Type Login
AND
(
184 = (Select Distinct Top 1 PermissionID from PermissionGroup (nolock) where GroupID in 
(Select GroupID from SiteUserGroup (nolock) where SiteUserID = @SiteUserID) and PermissionID = 184)
OR
A.AuditTypeID <> 20
)
 AND  A.PermissionID IN 
                ( SELECT PermissionID 
                FROM #TEMP
                 )

AND 
(
( 
UserID IN 
(
  SELECT Id FROM @ColleageUsers
)
)
)) AS A

ORDER BY A.id desc
OPTION(RECOMPILE,QUERYTRACEON 4138 );

Run Code Online (Sandbox Code Playgroud)

归档时间：	6 年前
查看次数：	162 次
最近记录：	5 年，12 月前