use*_*987 6 postgresql performance query-optimization greatest-n-per-group postgresql-performance
我正在尝试在表(玩家)和视图(player_main_colors)之间进行简单的连接:
SELECT P.*, C.main_color FROM players P
OUTER LEFT JOIN player_main_colors C USING (player_id)
WHERE P.user_id=1;
此查询大约需要40毫秒.
这里我在VIEW上使用嵌套的SELECT而不是JOIN:
SELECT player_id, main_color FROM player_main_colors
WHERE player_id IN (
SELECT player_id FROM players WHERE user_id=1);
此查询也需要约40毫秒.
当我将查询分成2个部分时,它会像我预期的那样变快:
SELECT player_id FROM players WHERE user_id=1;
SELECT player_id, main_color FROM player_main_colors
where player_id in (584, 9337, 11669, 12096, 13651,
13852, 9575, 23388, 14339, 500, 24963, 25630,
8974, 13048, 11904, 10537, 20362, 9216, 4747, 25045);
这些查询每个大约需要0.5毫秒.
那么为什么上面的查询与JOIN或子SELECT这么可怕的慢,我该如何修复呢？
以下是有关我的表格和视图的一些详细信息:
CREATE TABLE users (
user_id INTEGER PRIMARY KEY,
...
)
CREATE TABLE players (
player_id INTEGER PRIMARY KEY,
user_id INTEGER NOT NULL REFERENCES users (user_id),
...
)
CREATE TABLE player_data (
player_id INTEGER NOT NULL REFERENCES players (player_id),
game_id INTEGER NOT NULL,
color INTEGER NOT NULL,
PRIMARY KEY (player_id, game_id, color),
active_time INTEGER DEFAULT 0,
...
)
CREATE VIEW player_main_colors AS
SELECT DISTINCT ON (1) player_id, color as main_color
FROM player_data
GROUP BY player_id, color
ORDER BY 1, MAX(active_time) DESC
看来我的VIEW一定是个问题......？
这是上面嵌套的SELECT查询的EXPLAIN ANALYZE:
Merge Semi Join (cost=1877.59..2118.00 rows=6851 width=8) (actual time=32.946..38.471 rows=25 loops=1)
Merge Cond: (player_data.player_id = players.player_id)
-> Unique (cost=1733.19..1801.70 rows=13701 width=12) (actual time=32.651..37.209 rows=13419 loops=1)
-> Sort (cost=1733.19..1767.45 rows=13701 width=12) (actual time=32.646..34.918 rows=16989 loops=1)
Sort Key: player_data.player_id, (max(player_data.active_time))
Sort Method: external merge Disk: 376kB
-> HashAggregate (cost=654.79..791.80 rows=13701 width=12) (actual time=13.636..19.051 rows=17311 loops=1)
-> Seq Scan on player_data (cost=0.00..513.45 rows=18845 width=12) (actual time=0.005..1.801 rows=18845 loops=1)
-> Sort (cost=144.40..144.53 rows=54 width=8) (actual time=0.226..0.230 rows=54 loops=1)
Sort Key: players.player_id
Sort Method: quicksort Memory: 19kB
-> Bitmap Heap Scan on players (cost=4.67..142.85 rows=54 width=8) (actual time=0.035..0.112 rows=54 loops=1)
Recheck Cond: (user_id = 1)
-> Bitmap Index Scan on test (cost=0.00..4.66 rows=54 width=0) (actual time=0.023..0.023 rows=54 loops=1)
Index Cond: (user_id = 1)
Total runtime: 39.279 ms
至于索引,我只有一个相关的索引在我的主键的默认值之上:
CREATE INDEX player_user_idx ON players (user_id);
我目前正在使用PostgreSQL 9.2.9.
更新:
我已经减少了下面的问题.查看IN(4747)和IN(SELECT 4747)之间的差异.
慢:
>> explain analyze SELECT * FROM (
SELECT player_id, color
FROM player_data
GROUP BY player_id, color
ORDER BY MAX(active_time) DESC
) S
WHERE player_id IN (SELECT 4747);
Hash Join (cost=1749.99..1975.37 rows=6914 width=8) (actual time=30.492..34.291 rows=4 loops=1)
Hash Cond: (player_data.player_id = (4747))
-> Sort (cost=1749.95..1784.51 rows=13827 width=12) (actual time=30.391..32.655 rows=17464 loops=1)
Sort Key: (max(player_data.active_time))
Sort Method: external merge Disk: 376kB
-> HashAggregate (cost=660.71..798.98 rows=13827 width=12) (actual time=12.714..17.249 rows=17464 loops=1)
-> Seq Scan on player_data (cost=0.00..518.12 rows=19012 width=12) (actual time=0.006..1.898 rows=19012 loops=1)
-> Hash (cost=0.03..0.03 rows=1 width=4) (actual time=0.007..0.007 rows=1 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 1kB
-> HashAggregate (cost=0.02..0.03 rows=1 width=4) (actual time=0.006..0.006 rows=1 loops=1)
-> Result (cost=0.00..0.01 rows=1 width=0) (actual time=0.001..0.001 rows=1 loops=1)
Total runtime: 35.015 ms
(12 rows)
Time: 35.617 ms
快速:
>> explain analyze SELECT * FROM (
SELECT player_id, color
FROM player_data
GROUP BY player_id, color
ORDER BY MAX(active_time) DESC
) S
WHERE player_id IN (4747);
Subquery Scan on s (cost=17.40..17.45 rows=4 width=8) (actual time=0.035..0.035 rows=4 loops=1)
-> Sort (cost=17.40..17.41 rows=4 width=12) (actual time=0.034..0.034 rows=4 loops=1)
Sort Key: (max(player_data.active_time))
Sort Method: quicksort Memory: 17kB
-> GroupAggregate (cost=0.00..17.36 rows=4 width=12) (actual time=0.020..0.027 rows=4 loops=1)
-> Index Scan using player_data_pkey on player_data (cost=0.00..17.28 rows=5 width=12) (actual time=0.014..0.019 rows=5 loops=1)
Index Cond: (player_id = 4747)
Total runtime: 0.080 ms
(8 rows)
Time: 0.610 ms
你有两个GROUP BY和DISTINCT ON你的VIEW定义.这就像射杀了一个死人.简化:
CREATE VIEW player_main_colors AS
SELECT DISTINCT ON (1)
player_id, color AS main_color
FROM player_data
ORDER BY 1, active_time DESC NULLS LAST;
NULLS LAST必须等同于您的原始,因为active_time根据您的表定义可以为NULL.应该更快.但还有更多.为了获得最佳性能创建这些指标:
CREATE INDEX players_up_idx ON players (user_id, player_id);
CREATE INDEX players_data_pa_idx ON player_data
(player_id, active_time DESC NULLS LAST, color);
也可以DESC NULLS LAST在索引中使用以匹配查询的排序顺序.或者你改变player_data.active_time对NOT NULL和简化所有.
顺便说一句,它LEFT OUTER JOIN不是OUTER LEFT JOINOUTER:
SELECT * -- equivalent here to "p.*, c.main_color"
FROM players p
LEFT JOIN player_main_colors c USING (player_id)
WHERE p.user_id = 1;
我假设每个都有很多行.而你只是选择了一些.对于这种情况会是最快的,但你需要Postgres 9.3或更高版本.在第9.2页中,您可以使用相关子查询获得类似的效果:player_dataplayer_id player_idJOIN LATERAL
CREATE VIEW player_main_colors AS
SELECT player_id
, (SELECT color
FROM player_data
WHERE player_id = p.player_id
ORDER BY active_time DESC NULLS LAST
LIMIT 1) AS main_color
FROM players p
ORDER BY 1 -- optional
与原始视图的细微差别:这包括没有任何条目的玩家player_data.您可以根据新视图尝试与上面相同的查询.但我根本不会使用视图.这可能是最快的:
SELECT *
, (SELECT color
FROM player_data
WHERE player_id = p.player_id
ORDER BY active_time DESC NULLS LAST
LIMIT 1) AS main_color
FROM players p
WHERE p.user_id = 1;
详细说明:
因此，出现这种行为的原因是查询规划器存在局限性。在特定的绑定参数情况下，查询规划器能够根据它可以看到和分析的查询制定特定的计划。然而，当事情通过联接和子选择发生时，对将发生的情况的可见性要低得多。它使优化器使用更“通用”的计划 - 在这种情况下该计划的速度要慢得多。
对您来说正确的答案似乎是做出两个选择。也许更好的答案可能是将“main_color”非规范化到您的播放器表上并定期更新它。