为什么这个查询需要超过 1.3TB 的内存，它应该没有内存授权？

Question

SQL Server 版本是 2014 Developer SP1。跟踪标志272, 610, 1118, 1206, 1222, 8048, 9481是全局打开的。

我们不得不打开标志 9481，因为新的基数估计器严重影响了许多查询计划。查询没有排序、散列或并行。当我运行查询时，几分钟内没有返回结果。

在执行期间，sys.dm_exec_query_memory_grants dmv 还报告了超过 1.3TB 的理想内存。(WhyGiganticDesiredMemory.queryanalysis)

我运行了一个非常相似的查询。在 ActualPlan.queryanalysis 中所需的内存为 124GB，授予为 44GB，而在 ActualPlanWithLessMemoryGrant.queryanalysis 中所需的内存仅为 10MB。

它们之间的唯一区别是从后者中删除的嵌套循环下方的最左侧索引查找（我在连接条件中添加了一个无效过滤器 1 = 0 以从计划中删除该表）。

我的查询如下所示：

SELECT  CAST('C' AS CHAR(1)) AS [Action]
      , f.LoadId
      , u.UnvPclId
      , p.*    --- [about 480 columns, Daniel Hutmacher's edit]
FROM    [tExtract].[ExtractCounty] f
INNER JOIN [tControl].[VersionControl] vc ON Id = 1
INNER JOIN [tTax].[Property] pk WITH ( NOLOCK, FORCESEEK ( UpdateVersion_CntyCd ( UpdateVersion ) ) ) ON pk.UpdateVersion > vc.StartRowVersion
                                                                                                         AND pk.UpdateVersion <= vc.EndRowVersion
                                                                                                         AND pk.CntyCd = f.CntyCd
INNER JOIN [tTax].[Property] p WITH ( NOLOCK, FORCESEEK, INDEX = 1 ) ON p.[CntyCd] = pk.[CntyCd]
                                                                        AND p.[PclId] = pk.[PclId]
                                                                        AND p.[PclSeqNbr] = pk.[PclSeqNbr]
LEFT OUTER JOIN tCommon.UnvPclId u WITH ( NOLOCK, FORCESEEK ( 1 ( CntyCd, Edition, PclId, PclSeqNbr ) ) ) ON u.Edition = f.Edition
                                                                                                             AND u.CntyCd = f.CntyCd
                                                                                                             AND u.PclId = p.PclId
                                                                                                             AND u.PclSeqNbr = p.PclSeqNbr
                                                                                                             AND 1 = 0

WHERE   f.SchemaId = 1 /*tTax*/
        AND f.FullExtract = 0
        AND EXISTS ( SELECT 1
                     FROM   [tExtract].[TableNotCompleted] dt
                     WHERE  dt.TableId = 57 /*Property*/)

为什么GiganticDesiredMemory.queryanalysis

ActualPlanWithLessMemoryGrant.queryanalysis

实际计划查询分析

Answer 1

内存授予用于对嵌套循环连接进行预取和批量排序。请参阅“优化”和“ WithUnorderedPrefetch ”属性。

有记录的跟踪标志可以关闭优化- TF 2340；否则，优化器会根据基数估计做出决定。从 SQL Server 2016 SP1 开始，您还可以使用查询提示DISABLE_OPTIMIZED_NESTED_LOOP。

您可以调整查询以降低预期的行数，以便优化器决定不预取或批量排序。这样的重写也可以消除对这么多提示的“需要”。

更多信息：

• 嵌套循环预取
• 批量排序和嵌套循环

基本问题已在 SQL Server 2016 中得到解决，并将在某个阶段向后移植到 SQL Server 2014。来自 Pedro Lopes 的链接帖子：

...在 SQL Server 2016 RC0 中，我们更改了行为以保持优化的优势，但现在最大授予限制基于可用内存授予空间。

Answer 2

我的猜测是，当您在大型表上看到构建不当的执行计划时，很可能是您拥有过时的统计数据。当统计信息过期时，会使用错误的执行计划，这通常会严重影响 SQL Server 的性能。默认情况下，update auto stats 选项会在 20% + 500 条记录被修改时更新统计信息……因此，如果您有 1 亿条记录，它只会在 2000 万条记录被修改后更新统计信息，这可能需要很长时间（取决于在更新统计信息之前插入、更新和删除的频率。我最近遇到了一个类似的问题，我必须构建一个维护，使用一定比例的行进行采样来重新索引、重组和更新统计信息。

我的维护负责检查碎片和统计信息。我用它来检查索引的碎片并循环遍历每个结果：

SELECT t.name AS TableName, sch.name as SchemaName, i.name AS IndexName, s.avg_fragmentation_in_percent, ROW_NUMBER() OVER (PARTITION BY t.name order by t.name) as RowNum, TR.RowCnt
            FROM SYS.TABLES t
            INNER JOIN sys.schemas sch on t.schema_id=sch.schema_id
            JOIN SYS.INDEXES i ON t.object_id = i.object_id
            JOIN SYS.DM_DB_INDEX_PHYSICAL_STATS(DB_ID(),NULL,NULL,NULL,NULL) s ON t.object_id = s.object_id AND i.index_id = s.index_id
            LEFT JOIN (SELECT o.OBJECT_ID, ddps.row_count as RowCnt
                            FROM sys.indexes AS i
                            INNER JOIN sys.objects AS o ON i.OBJECT_ID = o.OBJECT_ID
                            INNER JOIN sys.dm_db_partition_stats AS ddps ON i.OBJECT_ID = ddps.OBJECT_ID
                            AND i.index_id = ddps.index_id
                            WHERE i.index_id < 2
                            AND o.is_ms_shipped = 0) TR ON TR.object_id = t.object_id
            WHERE t.type  = 'U'
            AND s.avg_fragmentation_in_percent > 5
            AND i.name is not null

然后用这个去遍历Statistics，更新需要更新的统计信息：

SELECT  OBJECT_NAME(sp.OBJECT_ID) AS TableName, SchemaName=sch.name,
                s.name AS StatName,
                CASE    WHEN pa.rows_in_table between 500 AND  500000 AND sp.modification_counter>(pa.rows_in_table*0.15) AND ISNULL(sp.last_updated,'1900-01-01')<DATEADD(DAY,-3,GETDATE()) then 100
                        WHEN pa.rows_in_table BETWEEN 500000 AND 1000000 AND sp.modification_counter>(pa.rows_in_table*0.10) AND ISNULL(sp.last_updated,'1900-01-01')<DATEADD(DAY,-3,GETDATE()) then 50
                        WHEN pa.rows_in_table BETWEEN 1000001 AND 5000000 AND sp.modification_counter>(pa.rows_in_table*0.05) AND ISNULL(sp.last_updated,'1900-01-01')<DATEADD(DAY,-3,GETDATE()) then 25
                        WHEN pa.rows_in_table BETWEEN 5000001 AND 10000000 AND sp.modification_counter>(pa.rows_in_table*0.025) AND ISNULL(sp.last_updated,'1900-01-01')<DATEADD(DAY,-3,GETDATE()) then 10
                        WHEN pa.rows_in_table BETWEEN 10000001 AND 50000000 AND sp.modification_counter>(pa.rows_in_table*0.02) AND ISNULL(sp.last_updated,'1900-01-01')<DATEADD(DAY,-3,GETDATE()) then 2
                        WHEN pa.rows_in_table BETWEEN 50000001 AND 100000000 AND sp.modification_counter>(pa.rows_in_table*0.01) AND ISNULL(sp.last_updated,'1900-01-01')<DATEADD(DAY,-3,GETDATE()) then 1
                        WHEN pa.rows_in_table>100000000 AND sp.modification_counter>(pa.rows_in_table*0.005) AND ISNULL(sp.last_updated,'1900-01-01')<DATEADD(DAY,-3,GETDATE()) then 3000000
                        ELSE 0 END StatsToUpdate
        FROM SYS.STATS AS s
        OUTER APPLY SYS.DM_DB_STATS_PROPERTIES(s.OBJECT_ID,s.stats_id) AS sp
        INNER JOIN (SELECT ta.OBJECT_ID, ta.schema_id, sum(pa.rows) rows_in_table
                        FROM SYS.TABLES ta
                        INNER JOIN SYS.PARTITIONS pa ON pa.OBJECT_ID = ta.OBJECT_ID
                        INNER JOIN SYS.SCHEMAS sc   ON ta.SCHEMA_ID = sc.SCHEMA_ID
                        WHERE ta.is_ms_shipped = 0 AND pa.index_id IN (1,0) 
                    GROUP BY ta.OBJECT_ID, ta.schema_id) as pa  ON pa.OBJECT_ID = s.OBJECT_ID
        INNER JOIN sys.schemas sch on pa.schema_id=sch.schema_id
        WHERE ISNULL(modification_counter,0) <> 0
        AND CASE    WHEN pa.rows_in_table between 500 AND  500000 AND sp.modification_counter>(pa.rows_in_table*0.15) AND ISNULL(sp.last_updated,'1900-01-01')<DATEADD(DAY,-3,GETDATE()) then 100
                        WHEN pa.rows_in_table BETWEEN 500000 AND 1000000 AND sp.modification_counter>(pa.rows_in_table*0.10) AND ISNULL(sp.last_updated,'1900-01-01')<DATEADD(DAY,-3,GETDATE()) then 50
                        WHEN pa.rows_in_table BETWEEN 1000001 AND 5000000 AND sp.modification_counter>(pa.rows_in_table*0.05) AND ISNULL(sp.last_updated,'1900-01-01')<DATEADD(DAY,-3,GETDATE()) then 25
                        WHEN pa.rows_in_table BETWEEN 5000001 AND 10000000 AND sp.modification_counter>(pa.rows_in_table*0.025) AND ISNULL(sp.last_updated,'1900-01-01')<DATEADD(DAY,-3,GETDATE()) then 10
                        WHEN pa.rows_in_table BETWEEN 10000001 AND 50000000 AND sp.modification_counter>(pa.rows_in_table*0.02) AND ISNULL(sp.last_updated,'1900-01-01')<DATEADD(DAY,-3,GETDATE()) then 2
                        WHEN pa.rows_in_table BETWEEN 50000001 AND 100000000 AND sp.modification_counter>(pa.rows_in_table*0.01) AND ISNULL(sp.last_updated,'1900-01-01')<DATEADD(DAY,-3,GETDATE()) then 1
                        WHEN pa.rows_in_table>100000000 AND sp.modification_counter>(pa.rows_in_table*0.005) AND ISNULL(sp.last_updated,'1900-01-01')<DATEADD(DAY,-3,GETDATE()) then 3000000
                        ELSE 0 END>0

请注意，我选择这些数字用于更新统计数据的抽样，但它们需要根据具体情况进行更改。