为时间戳记录创建适当的avro架构

koa*_*421 7 json data-conversion avro apache-nifi

我想知道适当的avro架构对于这种格式的一些json到avro转换是什么:

{"entryDate": "2018-01-26T12:00:40.930"}
Run Code Online (Sandbox Code Playgroud)

我的架构:

{
    "type" : "record",
    "name" : "schema",
    "fields" : [{
        "name" : "entryDate",
        "type" : ["null", {
            "type" : "long",
            "logicalType" : "timestamp-micros"
        }],
        "default" : null
    }]
}
Run Code Online (Sandbox Code Playgroud)

我一直在

`'Cannot convert field entryDate: Cannot resolve union: 
"2018-01-26T12:00:40.930" 
not in 
["null",{"type":"long","logicalType":"timestamp-millis"}]'`
Run Code Online (Sandbox Code Playgroud)

koa*_*421 7

这是一个愚蠢的错误......显然我将时间戳值存储为字符串,因此avro架构需要一个字符串而不是long类型.

即.

{
    "type" : "record",
    "name" : "schema",
    "fields" : [{
        "name" : "entryDate",
        "type" : ["null", {
            "type" : `**"long"**`,
            "logicalType" : "timestamp-micros"
        }],
        "default" : null
    }]
}
Run Code Online (Sandbox Code Playgroud)

应该

{
    "type" : "record",
    "name" : "schema",
    "fields" : [{
        "name" : "entryDate",
        "type" : ["null", {
            "type" : `**"string"**`,
            "logicalType" : "timestamp-micros"
        }],
        "default" : null
    }]
}
Run Code Online (Sandbox Code Playgroud)

卫生署!

  • 同样,有人知道字符串-> 时间/时间戳毫秒/微秒吗? (3认同)
  • 似乎逻辑类型在某种意义上受到限制,即它们只能注释某些基本类型.时间和日期的逻辑类型应仅注释数字类型,在timestamp-micros的情况下为"long"(参见[docs](https://avro.apache.org/docs/1.8.2/spec.html#Timestamp+) %28microsecond +精度%29)).你用`string`遇到了问题吗? (2认同)