如何在Elasticsearch中的嵌套和未嵌套日期之间执行日期算术？

Question

考虑以下Elasticsearch(v5.4)对象("奖励"doc类型):

{
  "name": "Gold 1000",
  "date": "2017-06-01T16:43:00.000+00:00",
  "recipient": {
    "name": "James Conroy",
    "date_of_birth": "1991-05-30"
  }
}

映射类型都award.date和award.recipient.date_of_birth是"日期".

我想进行范围汇总,以获得该奖项的获奖者年龄范围列表("18岁以下","18-24岁","24-30岁","30岁以上").奖励.我尝试了以下聚合查询:

{
  "size": 0,
  "query": {"match_all": {}},
  "aggs": {
    "recipients": {
      "nested": {
        "path": "recipient"
      },
      "aggs": {
        "age_ranges": {
          "range": {
            "script": {
              "inline": "doc['date'].date - doc['recipient.date_of_birth'].date"
            },
            "keyed": true,
            "ranges": [{
              "key": "Under 18",
              "from": 0,
              "to": 18
            }, {
              "key": "18-24",
              "from": 18,
              "to": 24
            }, {
              "key": "24-30",
              "from": 24,
              "to": 30
            }, {
              "key": "30+",
              "from": 30,
              "to": 100
            }]
          }
        }
      }
    }
  }
}

问题1

但由于script部分中的日期比较,我得到以下错误:

Cannot apply [-] operation to types [org.joda.time.DateTime] and [org.joda.time.MutableDateTime].

的DateTime对象是award.date字段,并且该MutableDateTime对象是award.recipient.date_of_birth字段.我尝试过做类似的事情doc['recipient.date_of_birth'].date.toDateTime()(尽管Joda文档声称MutableDateTime这个方法继承自父类,但它仍无效).我也尝试做过这样的事情:

"script": "ChronoUnit.YEARS.between(doc['date'].date, doc['recipient.date_of_birth'].date)"

遗憾的是也不起作用:(

问题2

我注意到我是这样做的:

"aggs": {
  "recipients": {
    "nested": {
      "path": "recipient"
    },
    "aggs": {
      "award_years": {
        "terms": {
          "script": {
            "inline": "doc['date'].date.year"
          }
        }
      }
    }
  }
}

我得到1970的doc_count结果恰好等于ES中的文档总数.这让我相信,访问嵌套对象之外的属性根本不起作用,并给了我一些默认值,如epoch datetime.如果我做相反的事情(汇总出生日期而没有筑巢),我会在所有出生日期得到完全相同的东西(1970年,纪元日期时间).那么如何比较这两个日期呢？

我在这里绞尽脑汁,我觉得有一些聪明的解决方案超出了我目前对Elasticsearch的专业知识.救命!

如果你想为此设置一个快速的环境来帮助我,这里有一些卷曲的好处:

curl -XDELETE http://localhost:9200/joelinux
curl -XPUT http://localhost:9200/joelinux -d "{\"mappings\": {\"award\": {\"properties\": {\"name\": {\"type\": \"string\"}, \"date\": {\"type\": \"date\", \"format\": \"yyyy-MM-dd'T'HH:mm:ss.SSSSSSZ\"}, \"recipient\": {\"type\": \"nested\", \"properties\": {\"name\": {\"type\": \"string\"}, \"date_of_birth\": {\"type\": \"date\", \"format\": \"yyyy-MM-dd\"}}}}}}}"
curl -XPUT http://localhost:9200/joelinux/award/1 -d '{"name": "Gold 1000", "date": "2016-06-01T16:43:00.000000+00:00", "recipient": {"name": "James Conroy", "date_of_birth": "1991-05-30"}}'
curl -XPUT http://localhost:9200/joelinux/award/2 -d '{"name": "Gold 1000", "date": "2017-02-28T13:36:00.000000+00:00", "recipient": {"name": "Martin McNealy", "date_of_birth": "1983-01-20"}}'

这应该会给你一个"joelinux"指数和两个"奖励"文档进行测试("James Conroy"和"Martin McNealy").提前致谢!

Answer 1

不幸的是，您无法在同一上下文中访问嵌套和非嵌套字段。作为解决方法，您可以使用以下选项更改映射以自动将日期从嵌套文档复制到根上下文copy_to：

{
    "mappings": {
        "award": {
            "properties": {
                "name": {
                    "fields": {
                        "keyword": {
                            "ignore_above": 256,
                            "type": "keyword"
                        }
                    },
                    "type": "text"
                },
                "date": {
                    "type": "date"
                },
                "date_of_birth": {
                    "type": "date" // will be automatically filled when indexing documents
                },
                "recipient": {
                    "properties": {
                        "name": {
                            "fields": {
                                "keyword": {
                                    "ignore_above": 256,
                                    "type": "keyword"
                                }
                            },
                            "type": "text"
                        },
                        "date_of_birth": {
                            "type": "date",
                            "copy_to": "date_of_birth" // copy value to root document
                        }
                    },
                    "type": "nested"
                }
            }
        }
    }
}

之后，您可以使用 path 访问出生日期date，尽管获取日期之间的年数的计算有点棘手：

Period.between(LocalDate.ofEpochDay(doc['date_of_birth'].date.getMillis() / 86400000L), LocalDate.ofEpochDay(doc['date'].date.getMillis() / 86400000L)).getYears()

这里我将原始JodaTime日期对象转换为system.time.LocalDate对象：

1. 获取 1970-01-01 以来的毫秒数
2. 除以 86400000L（一天中的毫秒数），转换为 1970-01-01 以来的天数
3. 转换为LocalDate对象
4. 从两个日期创建基于日期的Period对象
5. 获取两个日期之间的年数。

因此，最终的聚合查询如下所示：

{
    "size": 0,
    "query": {
        "match_all": {}
    },
    "aggs": {
        "age_ranges": {
            "range": {
                "script": {
                    "inline": "Period.between(LocalDate.ofEpochDay(doc['date_of_birth'].date.getMillis() / 86400000L), LocalDate.ofEpochDay(doc['date'].date.getMillis() / 86400000L)).getYears()"
                },
                "keyed": true,
                "ranges": [
                    {
                        "key": "Under 18",
                        "from": 0,
                        "to": 18
                    },
                    {
                        "key": "18-24",
                        "from": 18,
                        "to": 24
                    },
                    {
                        "key": "24-30",
                        "from": 24,
                        "to": 30
                    },
                    {
                        "key": "30+",
                        "from": 30,
                        "to": 100
                    }
                ]
            }
        }
    }
}