使用 levenshtein() 函数优化 PostgreSQL 查询

Question

我有一个包含大约 700 万条记录的表。该表有一个名字和姓氏列，我想使用 levenshtein() 距离函数进行搜索。

select levenshtein('JOHN', first_name) as fn_distance,
       levenshtein('DOE', last_name) as ln_distance,
       id,
       first_name as "firstName",
       last_name as "lastName"
  from person
 where first_name is not null
   and last_name is not null
   and levenshtein('JOHN', first_name) <= 2
   and levenshtein('DOE', last_name) <= 2
 order by 1, 2
 limit 50;

上面的搜索很慢（4 - 5 秒），我可以做些什么来提高性能？应该在两列上创建索引，还是其他什么？

添加以下索引后：

create index first_name_idx on person using gin (first_name gin_trgm_ops);

create index last_name_idx on person using gin(last_name gin_trgm_ops);

查询现在大约需要 11 秒。:(

新查询：

select similarity('JOHN', first_name) as fnsimilarity,
       similarity('DOW', last_name) as lnsimilarity,
       first_name as "firstName",
       last_name as "lastName",
       npi
  from person
 where first_name is not null
   and last_name is not null
   and similarity('JOHN', first_name) >= 0.2
   and similarity('DOW', last_name) >= 0.2
 order by 1 desc, 2 desc, npi
 limit 50;

Answer 1

没有支持编辑距离的内置索引类型。我也不知道有任何第三方索引实现可以这样做。

另一种字符串相似度度量，三元组相似度，确实有索引方法来支持它。也许您可以改用该措施。

您需要使用 % 运算符而不是相似度函数来编写查询。所以它看起来像这样：

set pg_trgm.similarity_threshold TO 0.2;
select similarity('JOHN', first_name) as fnsimilarity,
       similarity('DOW', last_name) as lnsimilarity,
       first_name as "firstName",
       last_name as "lastName",
       npi
  from person
 where first_name is not null
   and last_name is not null
   and 'JOHN' % first_name
   and 'DOW' % last_name
 order by 1, 2, npi
 limit 50;

但请注意，0.2 的截止值非常低，截止值越低，索引的效率就越低。