Elasticsearch上的JSONField解决方法：MapperParsingException

Question

如何JsonField将Django模型的Postgres映射到ElasticSearch索引？有什么解决方法可以使其正常工作？

参考：https : //github.com/sabricot/django-elasticsearch-dsl/issues/36

• models.py

class Web_Technology(models.Model):
    web_results = JSONField(blank=True,null=True,default=dict)

• web_results字段格式

{"http://google.com": {"Version": "1.0", "Server": "AkamaiGHost"}}

• documents.py

from elasticsearch_dsl import Index
from django_elasticsearch_dsl import Document, fields
from django_elasticsearch_dsl.registries import registry

from .models import Web_Technology

@registry.register_document
class WebTechDoc(Document):

    web_results = fields.ObjectField()

    def prepare_web_results(self, instance):
        return instance.web_results
    class Index:
        name = 'webtech'

    class Django:
        model = Web_Technology
        fields = []

`? python3 manage.py search_index --create -f
Creating index '<elasticsearch_dsl.index.Index object at 0x7f5f7b07ed30>'
Traceback (most recent call last):
  File "manage.py", line 15, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.5/dist-packages/django/core/management/__init__.py", line 381, in execute_from
_command_line
    utility.execute()
  File "/usr/local/lib/python3.5/dist-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.5/dist-packages/django/core/management/base.py", line 323, in run_from_argv
C    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.5/dist-packages/django/core/management/base.py", line 364, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.5/dist-packages/django_elasticsearch_dsl/management/commands/search_index.py", line 128, in handle
    self._create(models, options)
  File "/usr/local/lib/python3.5/dist-packages/django_elasticsearch_dsl/management/commands/search_index.py", line 84, in _create
    index.create()
  File "/usr/local/lib/python3.5/dist-packages/elasticsearch_dsl/index.py", line 254, in create
    self._get_connection(using).indices.create(index=self._name, body=self.to_dict(), **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/elasticsearch/client/utils.py", line 84, in _wrapped
    return func(*args, params=params, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/elasticsearch/client/indices.py", line 105, in create
    "PUT", _make_path(index), params=params, body=body
  File "/usr/local/lib/python3.5/dist-packages/elasticsearch/transport.py", line 350, in perform_request
    timeout=timeout,
  File "/usr/local/lib/python3.5/dist-packages/elasticsearch/connection/http_urllib3.py", line 252, in perform_request
    self._raise_error(response.status, raw_data)
  File "/usr/local/lib/python3.5/dist-packages/elasticsearch/connection/base.py", line 181, in _raise_error
    status_code, error_message, additional_info
elasticsearch.exceptions.RequestError: RequestError(400, 'MapperParsingException[mapping [properties]]; nested: MapperParsingException[Root type mapping not empty after parsing! Remaining fields:   [web_results : {type=object}]]; ', 'MapperParsingException[mapping [properties]]; nested: MapperParsingException[Root type mapping not empty after parsing! Remaining fields:   [web_results : {type=object}]]; ')

如果没有其他解决方法可以使它正常工作，请建议我使用另一个支持的快速搜索索引器JsonField。

ElasticSearch日志：

[2019-09-10 19:41:22,399][DEBUG][action.admin.indices.create] [cimexnode] [webtech] failed to create
org.elasticsearch.index.mapper.MapperParsingException: mapping [properties]
        at org.elasticsearch.cluster.metadata.MetaDataCreateIndexService$2.execute(MetaDataCreateIndexService.java:394)
        at org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.run(InternalClusterService.java:374)
        at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:204)
        at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:167)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.elasticsearch.index.mapper.MapperParsingException: Root type mapping not empty after parsing! Remaining fields:   [web_results : {type=object}]
        at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:278)
        at org.elasticsearch.index.mapper.DocumentMapperParser.parseCompressed(DocumentMapperParser.java:192)
        at org.elasticsearch.index.mapper.MapperService.parse(MapperService.java:449)
        at org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:307)
        at org.elasticsearch.cluster.metadata.MetaDataCreateIndexService$2.execute(MetaDataCreateIndexService.java:391)
        ... 6 more

Answer 1

如果您发布作品的链接（我没有测试它JSONField）中提到的方法，那么你就重写了错误的方法：在elasticsearch应用程序使用来准备场的方法prepare_FOO，其中FOO的字段名称。

因此，您无需调用方法prepare_web_results()，prepare_content_json()因为您的字段是web_results。现在，您的方法prepare_content_json将无用，因为它将永远不会被调用。

如果您的JSONField具有固定的结构，则应返回具有相应结构的object字段：

class WebTechDoc(Document):

    web_results = fields.ObjectField(properties={
        "url": fields.TextField(),
        "version": fields.TextField(),
        "server": fields.TextField()})

    def prepare_web_results(self, instance):
        results = instance.web_results
        url = results.keys()[0]
        return {
            "url": url,
            "version": results[url]["Version"],
            "server": results[url]["Server"]
        }

或者，如果您不太担心搜索结果的确切来源，则可以将字典映射到字符串，然后将其放在TextField()而不是中ObjectField()：return f"{instance.web_results}"