SQL INSERT INTO SELECT并将SELECT数据返回到Create Row View Counts

Question

所以我正在创建一个系统,它将从表中一次拉出50-150条记录并将它们显示给用户,并且我正在尝试为每条记录保留一个视图计数.

我认为最有效的方法是创建一个MEMORY表,我使用INSERT INTO将行的ID拉入,然后有一个cron函数定期运行以聚合视图ID计数并清除内存表,更新具有最新视图的原始计数.这样可以避免不断更新最可能被访问的表,因此我不会在每次查询时一次锁定150行(如果我使用的是MyISAM,则不是整个表).

基本上,这里解释的方法.

但是,我当然希望在我提取记录信息以供查看的同时执行此操作,并且我希望避免运行第二个单独的查询,以便为其计数获取相同的数据集.

有没有办法选择数据集,返回该数据集,同时将该数据集中的单个列插入另一个表？

看起来PostgreSQL可能有类似于我想要的RETURNING关键字,但我正在使用MySQL.

Answer 1

First of all, I would not add a counter column to the Main table. I would create a separate Audit table that would hold ID of the item from the Main table plus at least timestamp when that ID was requested. In essence, Audit table would store a history of requests. In this approach you can easily generate much more interesting reports. You can always calculate grand totals per item and also you can calculate summaries by day, week, month, etc per item or across all items. Depending on the volume of data you can periodically delete Audit entries older than some threshold (a month, a year, etc).

Also, you can easily store more information in Audit table as needed, for example, user ID to calculate stats per user.

To populate Audit table "automatically" I would create a stored procedure. The client code would call this stored procedure instead of performing the original SELECT. Stored procedure would return exactly the same result as original SELECT does, but would also add necessary details to the Audit table transparently to the client code.

So, let's assume that Audit table looks like this:

CREATE TABLE AuditTable
(
    ID int 
    IDENTITY -- SQL Server
    SERIAL -- Postgres
    AUTO_INCREMENT -- MySQL
    NOT NULL,
    ItemID int NOT NULL,
    RequestDateTime datetime NOT NULL
)

and your main SELECT looks like this:

SELECT ItemID, Col1, Col2, ...
FROM MainTable
WHERE <complex criteria>

To perform both INSERT and SELECT in one statement in SQL Server I'd use OUTPUT clause, in Postgres - RETURNING clause, in MySQL - ??? I don't think it has anything like this. So, MySQL procedure would have several separate statements.

MySQL

At first do your SELECT and insert results into a temporary (possibly memory) table. Then copy item IDs from temporary table into Audit table. Then SELECT from temporary table to return result to the client.

CREATE TEMPORARY TABLE TempTable
(
    ItemID int NOT NULL,
    Col1 ..., 
    Col2 ..., 
    ...
)
ENGINE = MEMORY
SELECT ItemID, Col1, Col2, ...
FROM MainTable
WHERE <complex criteria>
;

INSERT INTO AuditTable (ItemID, RequestDateTime)
SELECT ItemID, NOW()
FROM TempTable;

SELECT ItemID, Col1, Col2, ...
FROM TempTable
ORDER BY ...;

SQL Server (just to tease you. this single statement does both INSERT and SELECT)

MERGE INTO AuditTable
USING
(
    SELECT ItemID, Col1, Col2, ...
    FROM MainTable
    WHERE <complex criteria>
) AS Src
ON 1 = 0
WHEN NOT MATCHED BY TARGET THEN
INSERT
    (ItemID, RequestDateTime)
VALUES
    (Src.ItemID, GETDATE())
OUTPUT
    Src.ItemID, Src.Col1, Src.Col2, ...
;

You can leave Audit table as it is, or you can set up cron to summarize it periodically. It really depends on the volume of data. In our system we store individual rows for a week, plus we summarize stats per hour and keep it for 6 weeks, plus we keep daily summary for 18 months. But, important part, all these summaries are separate Audit tables, we don't keep auditing information in the Main table, so we don't need to update it.

Joe Celko explained it very well in SQL Style Habits: Attack of the Skeuomorphs:

Now go to any SQL Forum text search the postings. You will find thousands of postings with DDL that include columns named createdby, createddate, modifiedby and modifieddate with that particular meta data on the end of the row declaration. It is the old mag tape header label written in a new language! Deja Vu!

The header records appeared only once on a tape. But these meta data values appear over and over on every row in the table. One of the main reasons for using databases (not just SQL) was to remove redundancy from the data; this just adds more redundancy. But now think about what happens to the audit trail when a row is deleted? What happens to the audit trail when a row is updated? The trail is destroyed. The audit data should be separated from the schema. Would you put the log file on the same disk drive as the database? Would an accountant let the same person approve and receive a payment?