Elasticsearch自定义分析器无法正常工作

Question

我使用elasticsearch作为我的搜索引擎,我现在正在尝试创建一个自定义分析器,使字段值只是小写.以下是我的代码:

创建索引和映射

create index with a custom analyzer named test_lowercase?

curl -XPUT 'localhost:9200/test/' -d '{
  "settings": {
    "analysis": {
      "analyzer": {
        "test_lowercase": {
          "type": "pattern",
          "pattern": "^.*$"
        }
      }
    }
  }
}'

create a mapping using the test_lowercase analyzer for the address field?

curl -XPUT 'localhost:9200/test/_mapping/Users' -d '{
  "Users": {
    "properties": {
      "name": {
        "type": "string"
      },
      "address": {
        "type": "string",
        "analyzer": "test_lowercase"
      }
    }
  }
}'

验证test_lowercase分析器是否有效:

curl -XGET 'localhost:9200/test/_analyze?analyzer=test_lowercase&pretty' -d '
Beijing China
'
{
  "tokens" : [ {
    "token" : "\nbeijing china\n",
    "start_offset" : 0,
    "end_offset" : 15,
    "type" : "word",
    "position" : 0
  } ]
}

正如我们所看到的,字符串'北京中国'被编入单个小写的整个术语'beijing china',因此test_lowercase分析器工作正常.

要验证字段"地址"是否使用小写分析器:

curl -XGET 'http://localhost:9200/test/_analyze?field=address&pretty' -d '
Beijing China
'
{
  "tokens" : [ {
    "token" : "\nbeijing china\n",
    "start_offset" : 0,
    "end_offset" : 15,
    "type" : "word",
    "position" : 0
  } ]
}
curl -XGET 'http://localhost:9200/test/_analyze?field=name&pretty' -d '
Beijing China
'
{
  "tokens" : [ {
    "token" : "beijing",
    "start_offset" : 1,
    "end_offset" : 8,
    "type" : "<ALPHANUM>",
    "position" : 0
  }, {
    "token" : "china",
    "start_offset" : 9,
    "end_offset" : 14,
    "type" : "<ALPHANUM>",
    "position" : 1
  } ]
}

我们可以看到,对于相同的字符串'北京中国',如果我们使用field = address进行分析,它会创建一个单项'beijing china',当使用field = name时,我们得到两个项目'beijing'和'china' ,所以看来字段地址正在使用我的自定义分析器'test_lowercase'.

将文档插入测试索引以查看分析器是否适用于文档

curl -XPUT 'localhost:9200/test/Users/12345?pretty' -d '{"name": "Jinshui Tang",  "address": "Beijing China"}'

遗憾的是,该文档已成功插入,但地址字段尚未正确分析.我无法通过使用通配符查询来搜索它,如下所示:

curl -XGET 'http://localhost:9200/test/Users/_search?pretty' -d '
{
  "query": {
    "wildcard": {
      "address": "*beijing ch*"
    }
  }
}'
{
  "took" : 8,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 0,
    "max_score" : null,
    "hits" : [ ]
  }
}

列出为文档分析的所有术语:

所以我运行以下命令来查看文档的所有术语,我发现'北京中国'根本不在术语矢量中.

curl -XGET 'http://localhost:9200/test/Users/12345/_termvector?fields=*&pretty'
{
  "_index" : "test",
  "_type" : "Users",
  "_id" : "12345",
  "_version" : 3,
  "found" : true,
  "took" : 2,
  "term_vectors" : {
    "name" : {
      "field_statistics" : {
        "sum_doc_freq" : 2,
        "doc_count" : 1,
        "sum_ttf" : 2
      },
      "terms" : {
        "jinshui" : {
          "term_freq" : 1,
          "tokens" : [ {
            "position" : 0,
            "start_offset" : 0,
            "end_offset" : 7
          } ]
        },
        "tang" : {
          "term_freq" : 1,
          "tokens" : [ {
            "position" : 1,
            "start_offset" : 8,
            "end_offset" : 12
          } ]
        }
      }
    }
  }
}

我们可以看到这个名字被正确分析,它变成了两个术语'金水'和'唐',但地址丢失了.

有人可以帮忙吗？有什么遗漏？

非常感谢!

Answer 1

要小写文本,您不需要pattern.使用这样的东西:

PUT /test
{
  "settings": {
    "analysis": {
      "analyzer": {
        "test_lowercase": {
          "type": "custom",
          "filter": [
            "lowercase"
          ],
          "tokenizer": "keyword"
        }
      }
    }
  }
}

PUT /test/_mapping/Users
{
  "Users": {
    "properties": {
      "name": {
        "type": "string"
      },
      "address": {
        "type": "string",
        "analyzer": "test_lowercase"
      }
    }
  }
}

PUT /test/Users/12345
{"name": "Jinshui Tang",  "address": "Beijing China"}

为了验证你做对了,请使用:

GET /test/Users/_search
{
  "fielddata_fields": ["name", "address"]
}

你会看到究竟 Elasticsearch如何检索数据:

        "fields": {
           "name": [
              "jinshui",
              "tang"
           ],
           "address": [
              "beijing",
              "china"
           ]
        }