使用游标改进基于 SET 的方法的查询

Question

我仍然是查询优化的新手，我有一个存储过程，它使用游标遍历表中的每一行，并执行以下操作：

1. 计算每行之间的时间差
2. 计算每行之间的距离
3. 如果距离 < 5 AND TimeDifference > 3 分钟，则添加到 TEMP 表

我尝试将此 Cursor 转换为 WHILE 循环，但性能下降。所以我需要帮助将其转换为SET BASED方法而不是Procedural Based方法

所以 Cursor 执行这个逻辑：

-- READ Current Row into Cursor Variables
FETCH NEXT FROM crAssetIgnitionOnOff INTO
              @current_iVehicleMonitoringID
            , @current_iAssetID
            , @current_dtUTCDateTime
            , @current_sptGeoLocationPoint
            , @current_fLatitude
            , @current_fLongitude
            , @current_fAngle
            , @current_fSpeedKPH 
            , @current_sIgnitionStatus
            , @current_eEventCode
            , @current_sEventCode

IF(@current_iAssetID = @prev_iAssetID)
    BEGIN

        ---- Calculate Time Difference from previous Point
    DECLARE @diffInSeconds INT
        SET @diffInSeconds = DATEDIFF(SECOND, @prev_dtUTCDateTime, @current_dtUTCDateTime)

    DECLARE @diffInMinutes INT
        SET @diffInMinutes = @diffInSeconds / 60

        -- Calcualte the Distance from previous position
        DECLARE @tempDistance FLOAT;
            SELECT @tempDistance = @current_sptGeoLocaitonPoint.STDistance(@prev_sptGeoLocaitonPoint);

        -- Check if distance travelled less than 5, AND Time difference between points greater than user selected Idle Minutes (@iIdleMinutes) AND prev ignition status = On
        IF(@diffInSeconds > @iIdleMinutes AND @tempDistance < 5 AND @prev_sIgnitionStatus = 'On')   
            BEGIN

                DECLARE @sTime VARCHAR(30)
                    SELECT @sTime = dbo.xPT_ConvertTimeToDDHHMMSS(@diffInSeconds,'s')

                INSERT INTO @tblExcessiveIdleTime(
                                    AssetID,
                                    PreviousDate,
                                    CurrentDate,
                                    TimeString,
                                    TimeInSeconds
                            )
                            VALUES
                            (
                                @current_iAssetId,
                                @prev_dtUTCDateTime,
                                @current_dtUTCDateTime,
                                @sTime,
                                @diffInSeconds
                            )

            END 
    END


    -- Set Previous Values End of Loop
    SET  @prev_iVehicleMonitoringID = @current_iVehicleMonitoringID
    SET  @prev_iAssetID = @current_iAssetID
    SET  @prev_dtUTCDateTime = @current_dtUTCDateTime
    SET  @prev_sptGeoLocationPoint = @current_sptGeoLocationPoint
    SET  @prev_fLatitude = @current_fLatitude
    SET  @prev_fLongitude = @current_fLongitude
    SET  @prev_fAngle = @current_fAngle
    SET  @prev_fSpeedKPH = @current_fSpeedKPH
    SET  @prev_sIgnitionStatus = @current_sIgnitionStatus
    SET  @prev_eEventCode = @current_eEventCode
    SET  @prev_sEventCode = @current_sEventCode
END

现在在某些情况下执行需要 17 分钟，所以我尝试将其转换为 WHILE 循环 - ( http://www.sqlbook.com/SQL/Avoiding-using-SQL-Cursors-20.aspx )

这不是一个好主意 - 因为逻辑读取次数的性能是光标的 4 倍。处理时间更长：

WHILE @RowCount <= @NumberRecords
    BEGIN

        -- Check for First Row
        IF @RowCount = 1
            BEGIN
                -- Set First Row as Previous
                SELECT @previous_iAssetID = iAssetID, @previous_sptGeoLocaitonPoint = sptGeoLocaitonPoint, @previous_dtUTCDateTime = dtUTCDateTime, @previous_sIgnitionStatus = sIgnitionStatus
                FROM #tblVehicleMonitoringLog WHERE RowID = @RowCount
            END
        ELSE
            BEGIN

                /* Select current Row */
                SELECT @current_iAssetID = iAssetID, @current_sptGeoLocaitonPoint = sptGeoLocaitonPoint, @current_dtUTCDateTime = dtUTCDateTime, @current_sIgnitionStatus = sIgnitionStatus
                FROM #tblVehicleMonitoringLog WHERE RowID = @RowCount


                /******** IMPLEMENT REPORT LOGIC **********/

                IF(@current_iAssetID = @previous_iAssetID)
                    BEGIN

                        ---- Calculate Time Difference from previous Point
                    DECLARE @diffInSeconds INT
                        SET @diffInSeconds = DATEDIFF(SECOND, @previous_dtUTCDateTime, @current_dtUTCDateTime)

                    DECLARE @diffInMinutes INT
                        SET @diffInMinutes = @diffInSeconds / 60

                        -- Calcualte the Distance from previous position
                        DECLARE @tempDistance FLOAT;
                            SELECT @tempDistance = @current_sptGeoLocaitonPoint.STDistance(@previous_sptGeoLocaitonPoint);

                        -- Check if distance travelled less than 5, AND Time difference between points greater than user selected Idle Minutes (@iIdleMinutes) AND prev ignition status = On
                        IF(@diffInSeconds > @iIdleMinutes AND @tempDistance < 5 AND @previous_sIgnitionStatus = 'On')   
                            BEGIN

                                DECLARE @sTime VARCHAR(30)
                                    SELECT @sTime = dbo.xPT_ConvertTimeToDDHHMMSS(@diffInSeconds,'s')

                                INSERT INTO @tblExcessiveIdleTime(
                                                    iAssetID,
                                                    dtIgnitionOn,
                                                    dtNextPeriodic,
                                                    sTime,
                                                    iTimeDurationInSeconds
                                            )
                                            VALUES
                                            (
                                                @current_iAssetId,
                                                DATEADD(hour, @fGmtOffSet, @previous_dtUTCDateTime),
                                                DATEADD(hour, @fGmtOffSet, @current_dtUTCDateTime),
                                                @sTime,
                                                @diffInSeconds
                                            )

                            END 
                    END


                -- Set Previous Values End of Loop
                SET @previous_iAssetID = @current_iAssetID;
                SET @previous_sptGeoLocaitonPoint = @current_sptGeoLocaitonPoint;
                SET @previous_dtUTCDateTime = @current_dtUTCDateTime;
                SET @previous_sIgnitionStatus = @current_sIgnitionStatus;

            END



        -- increment Row Number
        SET @RowCount = @RowCount + 1


    END -- END OF WHILE LOOP

所以在再次在线查看后 - 我发现我们可以计算两行之间的时间差。（/sf/ask/165026081/）

这就是 RAW 数据的样子。

我需要计算每行之间的时差和距离 WHERE

1. 当前 iAssetID = 以前的 iAssetID
2. 上一个 sDigitalInputValue = '10000000'

这是我想出的查询：

WITH    rows AS
        (
            SELECT  *, ROW_NUMBER() OVER (ORDER BY dtUTCDateTime) AS rn
                FROM    VehicleMonitoringLog
                Where dtUTCDateTime > GetDate() - 1
                --Order by iAssetId, dtUTCDateTime
        )


SELECT mc.iVehicleMonitoringId as CurrentID, mp.iVehicleMonitoringId as PreviousID, 
        mc.iAssetId as CurrentAsset, mp.iAssetId As PreviousAsset, mc.dtUTCDateTime as CurrentTime, mp.dtUTCDateTime as PreviousTime,
 DATEDIFF(second, mc.dtUTCDateTime, mp.dtUTCDateTime) AS DateDiffSeconds
FROM    rows mc
JOIN    rows mp
ON      mc.rn = mp.rn - 1

编辑

我的查询现在正在运行 - 如果您发现任何性能问题，请告诉我：

SELECT dt.CurrentAsset,
        dt.Distance,
        dt.DateDiffSeconds,
        dt.CurrentIgnition,
        dt.PreviousIgnition,
        ta.sReference,
        ta.sCategoryName,
        ta.sSiteName,
    dbo.xPT_ConvertTimeToDDHHMMSS(DateDiffSeconds,'s')
FROM (
    SELECT iVehicleMonitoringId AS CurrentID,
       LEAD(iVehicleMonitoringId, 1) OVER (PARTITION BY iAssetID ORDER BY dtUTCDateTime) AS PreviousID, 
       iAssetId AS CurrentAsset,
       LEAD(iAssetId, 1) OVER (PARTITION BY iAssetID ORDER BY dtUTCDateTime) AS PreviousAsset,
       sDigitalInputValue AS CurrentIgnition,
       LEAD(sDigitalInputValue, 1) OVER (PARTITION BY iAssetID ORDER BY dtUTCDateTime) AS PreviousIgnition,
       dtUTCDateTime AS CurrentTime,
       LEAD(dtUTCDateTime, 1) OVER (PARTITION BY iAssetID ORDER BY dtUTCDateTime) AS PreviousTime,
       DATEDIFF(second, dtUTCDateTime, LEAD(dtUTCDateTime, 1) OVER (PARTITION BY iAssetID ORDER BY dtUTCDateTime)) AS DateDiffSeconds,
       sptGeoLocaitonPoint.STDistance(LEAD(sptGeoLocaitonPoint, 1) OVER (PARTITION BY iAssetID ORDER BY dtUTCDateTime)) AS Distance

        FROM VehicleMonitoringLog
        WHERE dtUTCDateTime > @utcStartDate AND dtUTCDateTime < @utcEndDate

) AS dt
Inner join #tblAssets ta on ta.iAssetID = dt.CurrentAsset
WHERE  CurrentIgnition = '10000000' AND Distance < 5 AND DateDiffSeconds > @iIdleMinutes

Answer 1

您使用窗口函数的基于 CTE 的方法是一个非常好的开始。您可以使用另一个更合适的窗口函数：LAG()。

就是这样：

SELECT iVehicleMonitoringId AS CurrentID,
       LAG(iVehicleMonitoringId, 1) OVER (ORDER BY dtUTCDateTime) AS PreviousID, 
       iAssetId AS CurrentAsset,
       LAG(iAssetId, 1) OVER (ORDER BY dtUTCDateTime) AS PreviousAsset,
       dtUTCDateTime AS CurrentTime,
       LAG(dtUTCDateTime, 1) OVER (ORDER BY dtUTCDateTime) AS PreviousTime,
       DATEDIFF(second,
           dtUTCDateTime,
           LAG(dtUTCDateTime, 1) OVER (ORDER BY dtUTCDateTime)
           ) AS DateDiffSeconds
FROM VehicleMonitoringLog
WHERE dtUTCDateTime > DATEADD(day, -1, SYSDATETIME());

基本上，LAG(column, n) OVER (ORDER BY x)返回column, nrow(s)的值（因此 n=1 返回前一行），按 排序x。

您的 CTE 解决方案将扫描VehicleMonitoringLog两次，然后加入两个流。此查询将只执行一次扫描，效率要高得多。该LAG()函数（及其表亲LEAD()）从 SQL Server 2012 开始可用。

分区

看起来您忘记了某种类型的分区术语（iAssetID，pehaps？）。分区项用于分隔不同车辆之间的数据点，以防两辆车同时出动。通过将 OVER() 子句从 更改为 ，将此分区项添加到查询OVER (ORDER BY dtUTCDateTime)中OVER (PARTITION BY iAssetID ORDER BY dtUTCDateTime)。

索引

为了使这个解决方案真正有效，我将在 上创建以下索引VehicleMonitoringLog：

CREATE INDEX... (iAssetID, dtUTCDateTime);   --- if you're using PARTITION BY

.. 或者

CREATE INDEX... (dtUTCDateTime);             --- without PARTITION BY