基于时间戳的窗口偏移

Question

基于时间戳的窗口偏移

Eri*_*rik 10 sql-server azure-sql-database window-functions paging

我正在编写一个查询，用于对社交提要的结果进行分页。这个概念是移动应用程序将请求 N 个项目，并提供我在@CutoffTime下面调用的开始日期时间。截止时间的目的是确定分页窗口应该何时开始。我们使用时间戳而不是行偏移量的原因是，即使添加了较新的社交内容，时间戳也会让我们在获取较旧帖子时从一致的位置翻页。

由于社交提要项目可以来自您自己或您的朋友，因此我使用 aUNION来组合这两个组的结果。最初我尝试了TheQuery_CTE没有的逻辑UNION，结果很慢。

这就是我所做的（包括相关的表架构）：

    CREATE TABLE [Content].[Photo]
(
    [PhotoId] INT NOT NULL PRIMARY KEY IDENTITY (1, 1), 
    [Key] UNIQUEIDENTIFIER NOT NULL DEFAULT NEWID(),
    [FullResolutionUrl] NVARCHAR(255) NOT NULL, 
    [Description] NVARCHAR(255) NULL, 
    [Created] DATETIME2(2) NOT NULL DEFAULT SYSUTCDATETIME(),
);

CREATE TABLE [Content].[UserPhotoAssociation]
(
    [PhotoId] INT NOT NULL, 
    [UserId] INT NOT NULL, 
    [ShowInSocialFeed] BIT NOT NULL DEFAULT 0,

    CONSTRAINT [PK_UserPhotos] PRIMARY KEY ([PhotoId], [UserId]), 
    CONSTRAINT [FK_UserPhotos_User] FOREIGN KEY ([UserId]) 
        REFERENCES [User].[User]([UserId]), 
    CONSTRAINT [FK_UserPhotos_Photo] FOREIGN KEY ([PhotoId]) 
        REFERENCES [Content].[Photo]([PhotoId])
);

CREATE TABLE [Content].[FlaggedPhoto]
(
    [FlaggedPhotoId] INT NOT NULL PRIMARY KEY IDENTITY(1,1),
    [PhotoId] INT NOT NULL,
    [FlaggedBy] INT NOT NULL,
    [FlaggedOn] DATETIME2(0) NOT NULL DEFAULT SYSDATETIME(),
    [FlaggedStatus] INT NOT NULL DEFAULT 1,
    [ReviewedBy] INT NULL,
    [ReviewedAt] DATETIME2(0) NULL

    CONSTRAINT [FK_Photos_PhotoId_to_FlaggedPhotos_PhotoId] FOREIGN KEY ([PhotoId]) 
        REFERENCES [Content].[Photo]([PhotoId]),
    CONSTRAINT [FK_FlaggedPhotoStatus_FlaggedPhotoStatusId_to_FlaggedPhotos_FlaggedStatus] FOREIGN KEY ([FlaggedStatus]) 
        REFERENCES [Content].[FlaggedContentStatus]([FlaggedContentStatusId]),
    CONSTRAINT [FK_User_UserId_to_FlaggedPhotos_FlaggedBy] FOREIGN KEY ([FlaggedBy]) 
        REFERENCES [User].[User]([UserId]),
    CONSTRAINT [FK_User_UserId_to_FlaggedPhotos_ReviewedBy] FOREIGN KEY ([ReviewedBy]) 
        REFERENCES [User].[User]([UserId])
);


CREATE TABLE [User].[CurrentConnections]
(
    [MonitoringId] INT NOT NULL PRIMARY KEY IDENTITY,
    [Monitor] INT NOT NULL,
    [Monitored] INT NOT NULL,
    [ShowInSocialFeed] BIT NOT NULL DEFAULT 1,

    CONSTRAINT [FK_Monitoring_Monitor_to_User_UserId] FOREIGN KEY ([Monitor]) 
         REFERENCES [dbo].[User]([UserId]),
    CONSTRAINT [FK_Monitoring_Monitored_to_User_UserId] FOREIGN KEY ([Monitored]) 
         REFERENCES [dbo].[User]([UserId])
);

CREATE TABLE [Content].[PhotoLike]
(
    [PhotoLikeId] INT NOT NULL PRIMARY KEY IDENTITY,
    [PhotoId] INT NOT NULL,
    [UserId] INT NOT NULL,
    [Created] DATETIME2(2) NOT NULL DEFAULT SYSUTCDATETIME(),
    [Archived] DATETIME2(2) NULL,

    CONSTRAINT [FK_PhotoLike_PhotoId_to_Photo_PhotoId] FOREIGN KEY ([PhotoId]) 
         REFERENCES [Content].[Photo]([PhotoId]),
    CONSTRAINT [FK_PhotoLike_UserId_to_User_UserId] FOREIGN KEY ([UserId]) 
         REFERENCES [User].[User]([UserId])
);

CREATE TABLE [Content].[Comment]
(
    [CommentId] INT NOT NULL PRIMARY KEY IDENTITY,
    [PhotoId] INT NOT NULL,
    [UserId] INT NOT NULL,
    [Comment] NVARCHAR(255) NOT NULL,
    [Created] DATETIME2(2) NOT NULL DEFAULT SYSUTCDATETIME(),
    [CommentOrder] DATETIME2(2) NOT NULL DEFAULT SYSUTCDATETIME(),
    [Archived] DATETIME2(2) NULL,

    CONSTRAINT [FK_Comment_PhotoId_to_Photo_PhotoId] FOREIGN KEY ([PhotoId]) 
         REFERENCES [Content].[Photo]([PhotoId]),
    CONSTRAINT [FK_Comment_UserId_to_User_UserId] FOREIGN KEY ([UserId]) 
         REFERENCES [User].[User]([UserId])
);

/*

      End table schema

*/



DECLARE @UserId INT,
    @NumberOfItems INT,
    @CutoffTime DATETIME2(2) = NULL -- Stored Proc input params

-- Make the joins and grab the social data we need once since they are used in subsequent queries that aren't shown
DECLARE @SocialFeed TABLE ([Key] UNIQUEIDENTIFIER, [PhotoId] INT
            , [Description] NVARCHAR(255), [FullResolutionUrl] NVARCHAR(255)
            , [Created] DATETIME2(2), [CreatorId] INT, [LikeCount] INT
            , [CommentCount] INT, [UserLiked] BIT);
-- Offset might be different for each group
DECLARE @OffsetMine INT = 0, @OffsetTheirs INT = 0;

IF @CutoffTime IS NOT NULL
    BEGIN
        -- Get the offsets
        ;WITH [GetCounts_CTE] AS
        (
            SELECT
                [P].[PhotoId] -- INT
                , 1 AS [MyPhotos]
            FROM [Content].[Photo] [P]
                INNER JOIN [Content].[UserPhotoAssociation] [UPA] ON 
                    [UPA].[PhotoId] = [P].[PhotoId] 
                    AND 
                    [UPA].[ShowInSocialFeed] = 1
                LEFT JOIN [Content].[FlaggedPhoto] [FP] ON 
                    [FP].[PhotoId] = [P].[PhotoId] 
                    AND 
                    [FP].[FlaggedStatus] = 3 -- Flagged photos that are confirmed apply to everyone
            WHERE
                [FP].[FlaggedPhotoId] IS NULL -- Filter out flagged photos
                AND
                [UPA].[UserId] = @UserId -- Show the requesting user
                AND
                [P].[Created] >= @CutoffTime -- Get the newer items
            UNION
            SELECT
                [P].[PhotoId] -- INT
                , 0 AS [MyPhotos]
            FROM [Content].[Photo] [P]
                INNER JOIN [Content].[UserPhotoAssociation] [UPA] ON 
                    [UPA].[PhotoId] = [P].[PhotoId] 
                    AND 
                    [UPA].[ShowInSocialFeed] = 1
                INNER JOIN [User].[CurrentConnections] [M] ON 
                    [M].[Monitored] = [UPA].[UserId] 
                    AND 
                    [M].[Monitor] = @UserId AND [M].[ShowInSocialFeed] = 1 -- this join isn't present above  
                LEFT JOIN [Content].[FlaggedPhoto] [FP] ON 
                    [FP].[PhotoId] = [P].[PhotoId] 
                    AND 
                    (
                        [FP].[FlaggedStatus] = 3 
                        OR 
                        ([FP].[FlaggedBy] = @UserId AND [FP].[FlaggedStatus] = 1)
                    ) -- Flagged photos that are confirmed apply to everyone, pending flags apply to the user
            WHERE
                [FP].[FlaggedPhotoId] IS NULL -- Filter out flagged photos
                AND
                [P].[Created] >= @CutoffTime -- Get the newer items
        )
        SELECT
            @OffsetMine = SUM(CASE WHEN [MyPhotos] = 1 THEN 1 ELSE 0 END)
            , @OffsetTheirs = SUM(CASE WHEN [MyPhotos] = 0 THEN 1 ELSE 0 END)
        FROM [GetCounts_CTE]
    END

-- Prevent absence of social data from throwing an error below.
SET @OffsetMine = ISNULL(@OffsetMine, 0); 
SET @OffsetTheirs = ISNULL(@OffsetTheirs, 0);

-- Actually select the data I want
;WITH TheQuery_CTE AS
(
    SELECT
        [P].[Key]
        , [P].[PhotoId]
        , [P].[Description]
        , [P].[FullResolutionUrl]
        , [P].[Created]
        , [UPA].[UserId]
        , COUNT(DISTINCT [PL].[PhotoLikeId]) AS [LikeCount] -- Count distinct used due to common join key
        , COUNT(DISTINCT [C].[CommentId]) AS [CommentCount]
        , CAST(ISNULL(MAX(CASE WHEN [PL].[UserId] = @UserId THEN 1 END), 0) AS BIT) AS [UserLiked]
    FROM [Content].[Photo] [P]
        INNER JOIN [Content].[UserPhotoAssociation] [UPA] ON 
            [UPA].[PhotoId] = [P].[PhotoId] 
            AND 
            [UPA].[ShowInSocialFeed] = 1
        LEFT JOIN [Content].[PhotoLike] [PL] ON 
            [PL].[PhotoId] = [P].[PhotoId] 
            AND 
            [PL].[Archived] IS NULL
        LEFT JOIN [Content].[Comment] [C] ON 
            [C].[PhotoId] = [P].[PhotoId] 
            AND 
            [C].[Archived] IS NULL
        LEFT JOIN [Content].[FlaggedPhoto] [FP] ON 
            [FP].[PhotoId] = [P].[PhotoId] 
            AND 
            [FP].[FlaggedStatus] = 3 -- Flagged photos that are confirmed apply to everyone
    WHERE
        [FP].[FlaggedPhotoId] IS NULL -- Filter out flagged photos
        AND
        [UPA].[UserId] = @UserId -- Show the requesting user
    GROUP BY
        [P].[Key]
        , [P].[PhotoId]
        , [P].[Description]
        , [P].[FullResolutionUrl]
        , [P].[Created]
        , [UPA].[UserId]
    ORDER BY  
        [P].[Created] DESC
        , [P].[Key]  -- Ensure consistent order in case of duplicate timestamps
        OFFSET @OffsetMine ROWS FETCH NEXT @NumberOfItems ROWS ONLY
    UNION
    SELECT
        [P].[Key]
        , [P].[PhotoId]
        , [P].[Description]
        , [P].[FullResolutionUrl]
        , [P].[Created]
        , [UPA].[UserId]
        , COUNT(DISTINCT [PL].[PhotoLikeId]) AS [LikeCount]
        , COUNT(DISTINCT [C].[CommentId]) AS [CommentCount]
        , CAST(ISNULL(MAX(CASE WHEN [PL].[UserId] = @UserId THEN 1 END), 0) AS BIT) AS [UserLiked]
    FROM [Content].[Photo] [P]
        INNER JOIN [Content].[UserPhotoAssociation] [UPA] ON 
            [UPA].[PhotoId] = [P].[PhotoId] 
            AND 
            [UPA].[ShowInSocialFeed] = 1
        INNER JOIN [User].[CurrentConnections] [M] ON 
            [M].[Monitored] = [UPA].[UserId] 
            AND 
            [M].[Monitor] = @UserId AND [M].[ShowInSocialFeed] = 1
        LEFT JOIN [Content].[PhotoLike] [PL] ON 
            [PL].[PhotoId] = [P].[PhotoId] 
            AND 
            [PL].[Archived] IS NULL
        LEFT JOIN [Content].[Comment] [C] ON 
            [C].[PhotoId] = [P].[PhotoId] 
            AND 
            [C].[Archived] IS NULL
        LEFT JOIN [Content].[FlaggedPhoto] [FP] ON 
            [FP].[PhotoId] = [P].[PhotoId] 
            AND 
            (
                [FP].[FlaggedStatus] = 3 
                OR 
                ([FP].[FlaggedBy] = @UserId AND [FP].[FlaggedStatus] = 1)
            ) -- Flagged photos that are confirmed apply to everyone, pending flags apply to the user
    WHERE
        [FP].[FlaggedPhotoId] IS NULL -- Filter out flagged photos
    GROUP BY
        [P].[Key]
        , [P].[PhotoId]
        , [P].[Description]
        , [P].[FullResolutionUrl]
        , [P].[Created]
        , [UPA].[UserId]
    ORDER BY  
        [P].[Created] DESC
        , [P].[Key]  -- Ensure consistant order in case of duplicate timestamps
        OFFSET @OffsetTheirs ROWS FETCH NEXT @NumberOfItems ROWS ONLY
)
INSERT INTO @SocialFeed ([Key], [PhotoId], [Description], [FullResolutionUrl]
            , [Created], [CreatorId], [LikeCount], [CommentCount], [UserLiked])
SELECT TOP (@NumberOfItems)
    [Key]
    , [PhotoId]
    , [Description]
    , [FullResolutionUrl]
    , [Created]
    , [UserId]
    , [LikeCount]
    , [CommentCount]
    , [UserLiked]
FROM [TheQuery_CTE]
ORDER BY  -- Order here so the top works properly
    [Created] DESC
    , [Key]  -- Ensure consistent order in case of duplicate timestamps

-- Output the social feed
SELECT
    [P].[Key]
    , [P].[PhotoId]
    , [P].[Description] AS [PhotoDescription]
    , [P].[FullResolutionUrl]
    , [P].[Created] AS [Posted]
    , [P].[CreatorId]
    , [LikeCount]
    , [CommentCount]
    , [UserLiked]
FROM @Photos [P]

-- Select other data needed to build the object tree in the application layer

Run Code Online (Sandbox Code Playgroud)

我意识到我可以摆脱UNIONin ，GetCounts_CTE但我认为它不会真正解决我在下面看到的任何问题。

我看到了一些潜在的问题：

这是很多重复的逻辑，所以我可能让自己的生活变得更艰难。
如果在计算计数和选择数据之间发生插入，我将关闭。我认为这不会经常发生，但会导致奇怪/难以调试的错误。
通过上述设置，人们会发现所有问题更聪明/更有经验。

编写此查询的最佳方法是什么？奖励积分解决方案让我的生活更简单。

编辑：

我不想选择所有数据并让客户端懒惰地显示项目，因为我不想通过强迫他们下载他们永远不会看到的项目来滥用人们的数据计划。诚然，从宏观计划来看，数据可能不会那么大，但一分钱一分货......

编辑2：

我强烈怀疑这不是最佳解决方案，但它是迄今为止我想出的最好的解决方案。

将我的UNION查询移动到Greg建议的VIEW类似方法可以很好地隐藏该逻辑并在我的存储过程中提供更简洁的查询。该视图还抽象了联合的丑陋/复杂性，这很好，因为我在我的选择中使用了两次。这是视图的代码：

CREATE VIEW [Social].[EverFeed]
    AS 
SELECT
    [P].[Key]
    , [P].[PhotoId]
    , [P].[Description]
    , [P].[FullResolutionUrl]
    , [P].[Created]
    , [UPA].[UserId]
    , COUNT(DISTINCT [PL].[PhotoLikeId]) AS [LikeCount] -- Distinct due to common join key
    , COUNT(DISTINCT [C].[CommentId]) AS [CommentCount]
    , CAST(ISNULL(
        MAX(CASE WHEN [PL].[UserId] = [UPA].[UserId] THEN 1 END), 0) AS BIT) AS [UserLiked]
    , NULL AS [Monitor]
FROM [Content].[Photo] [P]
    INNER JOIN [Content].[UserPhotoAssociation] [UPA] ON 
        [UPA].[PhotoId] = [P].[PhotoId] 
        AND 
        [UPA].[ShowInSocialFeed] = 1
    LEFT JOIN [Content].[PhotoLike] [PL] ON 
        [PL].[PhotoId] = [P].[PhotoId] 
        AND 
        [PL].[Archived] IS NULL
    LEFT JOIN [Content].[Comment] [C] ON 
        [C].[PhotoId] = [P].[PhotoId] 
        AND 
        [C].[Archived] IS NULL
    LEFT JOIN [Content].[FlaggedPhoto] [FP] ON 
        [FP].[PhotoId] = [P].[PhotoId] 
        AND 
        [FP].[FlaggedStatus] = 3 -- Flagged photos that are confirmed apply to everyone
WHERE
    [FP].[FlaggedPhotoId] IS NULL -- Filter out flagged photos
GROUP BY
    [P].[Key]
    , [P].[PhotoId]
    , [P].[Description]
    , [P].[FullResolutionUrl]
    , [P].[Created]
    , [UPA].[UserId]
UNION
SELECT
    [P].[Key]
    , [P].[PhotoId]
    , [P].[Description]
    , [P].[FullResolutionUrl]
    , [P].[Created]
    , [UPA].[UserId]
    , COUNT(DISTINCT [PL].[PhotoLikeId]) AS [LikeCount]
    , COUNT(DISTINCT [C].[CommentId]) AS [CommentCount]
    , CAST(ISNULL(
        MAX(CASE WHEN [PL].[UserId] = [M].[Monitor] THEN 1 END), 0) AS BIT) AS [UserLiked]
    , [M].[Monitor]
FROM [Content].[Photo] [P]
    INNER JOIN [Content].[UserPhotoAssociation] [UPA] ON 
        [UPA].[PhotoId] = [P].[PhotoId] 
        AND 
        [UPA].[ShowInSocialFeed] = 1
    INNER JOIN [User].[CurrentConnections] [M] ON 
        [M].[Monitored] = [UPA].[UserId] 
        AND 
        [M].[ShowInSocialFeed] = 1
    LEFT JOIN [Content].[PhotoLike] [PL] ON 
        [PL].[PhotoId] = [P].[PhotoId] 
        AND 
        [PL].[Archived] IS NULL
    LEFT JOIN [Content].[Comment] [C] ON 
        [C].[PhotoId] = [P].[PhotoId] 
        AND 
        [C].[Archived] IS NULL
    LEFT JOIN [Content].[FlaggedPhoto] [FP] ON 
        [FP].[PhotoId] = [P].[PhotoId] 
        AND 
        (
            [FP].[FlaggedStatus] = 3 
            OR 
            ([FP].[FlaggedBy] = [M].[Monitor] AND [FP].[FlaggedStatus] = 1)
        ) -- Flagged photos that are confirmed (3) apply to everyone
          -- , pending flags (1) apply to the user
WHERE
    [FP].[FlaggedPhotoId] IS NULL -- Filter out flagged photos
GROUP BY
    [P].[Key]
    , [P].[PhotoId]
    , [P].[Description]
    , [P].[FullResolutionUrl]
    , [P].[Created]
    , [UPA].[UserId]
    , [M].[Monitor]

Run Code Online (Sandbox Code Playgroud)

使用该视图，我将查询缩短为以下内容。注意我正在OFFSET使用子查询设置。

DECLARE @UserId INT, @NumberOfItems INT, @CutoffTime DATETIME2(2);

SELECT
    [Key]
    , [PhotoId]
    , [Description]
    , [FullResolutionUrl]
    , [Created]
    , [UserId]
    , [LikeCount]
    , [CommentCount]
    , [UserLiked]
FROM  [Social].[EverFeed] [EF]
WHERE
    (
        ([EF].[UserId] = @UserId AND [EF].[Monitor] IS NULL)
        OR 
        [EF].[Monitor] = @UserId
    )
ORDER BY  -- Order here so the top works properly
    [Created] DESC
    , [Key]  -- Ensure consistant order in case of duplicate timestamps
    OFFSET CASE WHEN @CutoffTime IS NULL THEN 0 ELSE        
            (
                SELECT
                    COUNT([PhotoId])
                FROM [Social].[EverFeed] [EF]
                WHERE
                    (
                        ([EF].[UserId] = @UserId AND [EF].[Monitor] IS NULL)
                        OR 
                        [EF].[Monitor] = @UserId
                    )
                    AND
                    [EF].[Created] >= @CutoffTime -- Get the newer items
            ) END 
    ROWS FETCH NEXT @NumberOfItems ROWS ONLY

Run Code Online (Sandbox Code Playgroud)

该视图很好地将UNION过滤器的复杂性区分开来。 我认为OFFSET子句中的子查询将通过使整个查询原子化来防止我担心的并发问题。

我在输入时发现的一个问题是：在上面的代码中，如果具有相同创建日期的两张照片在不同的“页面”上，那么后续页面上的照片将被过滤掉。考虑以下数据：

PhotoId | Created | ...
------------------------
   1    | 2015-08-26 01:00.00
   2    | 2015-08-26 01:00.00
   3    | 2015-08-26 01:00.00

Run Code Online (Sandbox Code Playgroud)

初始页面上的页面大小为 1 时，PhotoId 1将返回。如果第二页上的页面大小相同，则不会返回任何结果。我想为了解决这个问题，我将不得不将KeyGuid添加为参数....

Answer 1

Mik*_*son 5

我建议你走一条不同的道路。看起来一个模式select top() .... order by应该就足够了。

select top(@NumberOfItems)
  P.PhotoId,
  P.[Key],
  P.Created
from Content.Photo as P
  inner join Content.UserPhotoAssociation as UPA
    on P.PhotoId = UPA.PhotoId
where
  -- Older than CutoffTime
  P.Created < @CutoffTime and
  (
  -- My photos
  UPA.UserId = @UserId or
  -- Photos by someone monitored by me
  UPA.UserID in (
                select CC.Monitored
                from [User].CurrentConnections as CC
                where CC.Monitor = @UserId
                )
  )
order by P.Created desc, P.[Key]

Run Code Online (Sandbox Code Playgroud)

要处理 Created 恰好相等的情况，您需要Key在截止处理中包含该值。也许像这样的 where 子句。

P.Created < @CutoffTime or (P.Created = @CutoffTime and P.Key > @CutoffKey)

Run Code Online (Sandbox Code Playgroud)

归档时间：	10 年，2 月前
查看次数：	255 次
最近记录：	10 年，2 月前