我最近开始研究 jolt 库，它在 json 到 json 转换方面非常好。但我面临一些问题。其中之一是我无法在多个地方使用特定字段。例如：输入文件包含时间戳字段，输出是一个 json 数组。我想将其复制到每个数组项中。任何人都可以在这里提供帮助。输入和输出还将包含其他字段。
输入：{“timeStamp”：“21838188137282”}输出：[{“time”：“21838188137282”}，{“time”：“21838188137282”}{“time”：“21838188137282”}]
请帮助我编写颠簸规范，以获得带有特殊字符“@”的预期输出，如下所示。当我将 "@FirstName":"Names.FirstName" 放入规范中时，它不起作用。请帮助解决这个问题。
JSON 输入：
{
"FirstName": "First"
}
震动规格：
[
{
"operation": "shift",
"spec": {
"FirstName": "Names.FirstName"
}
}
]
预期输出：
{
"Names" : {
"@FirstName" : "First"
}
}
我想使用JOLT转换库重命名嵌套在另一个数组中的数组中的字段.1.要重命名的一个字段是数组2中的顶级字段.要重命名的两个字段位于嵌套数组中
我尝试过使用通配符,但是他们没有给我预期的输出.我使用的是JOLT 0.0.22版本.
输入JSON:
{
"country": "usa",
"state": [
{
"stateName": "TX",
"location": "south",
"cities": [
{
"name": "Austin",
"pop": "1M"
},
{
"name": "Dallas",
"pop": "2M"
}
]
},
{
"stateName": "CA",
"location": "west",
"cities": [
{
"name": "SanFran",
"pop": "3M"
},
{
"name": "LosAngeles",
"pop": "4M"
}
]
}
]
}
预期产出:
{
"country": "usa",
"state": [
{
"stateName": "TX",
"locatedIn": "south", // name change here
"cities": [
{
"cityname": "Austin", // name change here
"citypopulation": "1M" // …我正在尝试使用Jolt转换非常简单,但努力让它发挥作用.
如果我有一个输入,如:
{
"id": "54436001"
}
我希望输出为:
{
"mediaId" : "54436001",
"events" : {
"mediaId" : "54436001"
}
}
这是将值复制到两个不同的属性.我很想尝试使用这样的规格来工作,但很明显它并不是因为重复的密钥.
[
{
"operation": "shift",
"spec": {
"id": "mediaId",
"id": "events.mediaId"
}
}
]
这可能与Jolf转换有关吗？
我想使用 Jolt 处理器实现 JSON 转换。我的 JSON 中有包含 null 的字段，我想删除所有这些字段...
{
"myValue": 345,
"colorValue": null,
"degreeDayValue": null,
"depthValue": null,
"distanceValue": null
}
...只是为了保留这个myValue领域。
我可以通过删除 Jolt 操作来实现这一点吗？
Input
{
"name": "Karan",
"age": 25,
"contact": {
"email": "abc@gmail.com"
},
"details": [
{
"contact": {
"firstName": "karan",
"lastName": "singh"
},
"phone": "5555555555",
"email": "karan@gmail.com",
"address": {
"line1": "123",
"city": "bangalore",
"state": "karnataka",
"country": "india",
"zip": "570089"
}
}
],
"shippingTo": {
"contact": {
"name": {
"firstName": "5505",
"lastName": "5505"
},
"phone": null,
"email": null
},
"address": {
"line1": "2100 88th St",
"city": "Mumbai",
"state": "Maharashtra",
"country": "India",
"zip": "07047"
}
},
"fulfillmentLines": [
{
"chargeDetails": [
{
"chargeCategory": "PRODUCT", …我必须转换的输入JSON如下:
{
"Business": [
{
"Label": "Entertainment",
"category": "Advert",
"weight": "",
"types": [
"T1",
"T2"
]
},
{
"Label": "FMCG",
"category": "Campaign",
"weight": "",
"types": [
"T9",
"T10"
]
}
]
}
预期产出:
{
"Business": [
{
"Label": "Entertainment",
"category": "Advert",
"weight": "",
"types": "T1"
},
{
"Label": "Entertainment",
"category": "Advert",
"weight": "",
"types": "T2"
},
{
"Label": "FMCG",
"category": "Campaign",
"weight": "",
"types": "T9"
},
{
"Label": "FMCG",
"category": "Campaign",
"weight": "",
"types": "T10"
}
]
}
我已经尝试了JOLT github帮助页面提供的不同JsonSpecs.但我无法解决这个问题.任何帮助或指示将不胜感激.
是否可以使用 JOLT SPEC 将值映射到键。我的输入 Json
{
"name": "Rj",
"Age" : "15"
}
Output Json as
{
"Rj" : {
"Age":"15"
}
}
请为上述场景提供 Json SPEC
我正在从第三方获得CSV文件。该文件的模式是动态的，我唯一可以确定的是，
现在要在我的系统中使用这种类型的数据，我正在考虑将MongoDB用作登台区域。由于没有 从一次加载到另一次加载，列的数量，列的顺序或列的名称都不恒定。我认为MongoDB将成为一个很好的过渡区域。
我读了有关ConvertRecord处理器的信息，该处理器非常适合CSV到JSON转换器，但没有架构。我只希望每行都作为一个文档，标题名称作为键，值作为值。
我应该怎么做？另外，该文件的大小约为25-30 GB，因此我不想降低系统的性能。
我想到了用我自己的处理器（用Java）进行处理，并且能够得到所需的内容，但是这似乎花费了太多时间，而且看起来并不是最佳选择。
让我知道，这是否可以通过现有处理器实现？
谢谢，Rakesh
更新日期：2018/09/05
<?xml version="1.0" encoding="UTF-8" standalone="yes"?><template encoding-version="1.2"><description></description><groupId>a2bd0551-0165-1000-7c6a-a32ca4db047c</groupId><name>csv_to_json_no_schema_v1</name><snippet><connections><id>91bc4a66-704c-3a2f-0000-000000000000</id><parentGroupId>defb04c4-c15c-3a07-0000-000000000000</parentGroupId><backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold><backPressureObjectThreshold>10000</backPressureObjectThreshold><destination><groupId>defb04c4-c15c-3a07-0000-000000000000</groupId><id>bb6c25ae-f2b6-386a-0000-000000000000</id><type>PROCESSOR</type></destination><flowFileExpiration>0 sec</flowFileExpiration><labelIndex>1</labelIndex><name></name><selectedRelationships>success</selectedRelationships><source><groupId>defb04c4-c15c-3a07-0000-000000000000</groupId><id>eb6cd54a-e1f1-3871-0000-000000000000</id><type>PROCESSOR</type></source><zIndex>0</zIndex></connections><connections><id>ad804e3c-f233-3556-0000-000000000000</id><parentGroupId>defb04c4-c15c-3a07-0000-000000000000</parentGroupId><backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold><backPressureObjectThreshold>10000</backPressureObjectThreshold><destination><groupId>defb04c4-c15c-3a07-0000-000000000000</groupId><id>64b15a56-8a5f-3297-0000-000000000000</id><type>PROCESSOR</type></destination><flowFileExpiration>0 sec</flowFileExpiration><labelIndex>1</labelIndex><name></name><selectedRelationships>invalid</selectedRelationships><source><groupId>defb04c4-c15c-3a07-0000-000000000000</groupId><id>bb6c25ae-f2b6-386a-0000-000000000000</id><type>PROCESSOR</type></source><zIndex>0</zIndex></connections><connections><id>c30bd123-c436-36ce-0000-000000000000</id><parentGroupId>defb04c4-c15c-3a07-0000-000000000000</parentGroupId><backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold><backPressureObjectThreshold>10000</backPressureObjectThreshold><destination><groupId>defb04c4-c15c-3a07-0000-000000000000</groupId><id>8a0e37da-acd2-3d72-0000-000000000000</id><type>PROCESSOR</type></destination><flowFileExpiration>0 sec</flowFileExpiration><labelIndex>1</labelIndex><name></name><selectedRelationships>valid</selectedRelationships><source><groupId>defb04c4-c15c-3a07-0000-000000000000</groupId><id>bb6c25ae-f2b6-386a-0000-000000000000</id><type>PROCESSOR</type></source><zIndex>0</zIndex></connections><connections><id>247d2139-26b7-31fe-0000-000000000000</id><parentGroupId>defb04c4-c15c-3a07-0000-000000000000</parentGroupId><backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold><backPressureObjectThreshold>10000</backPressureObjectThreshold><destination><groupId>defb04c4-c15c-3a07-0000-000000000000</groupId><id>1297bea9-b30f-3f45-0000-000000000000</id><type>PROCESSOR</type></destination><flowFileExpiration>0 sec</flowFileExpiration><labelIndex>1</labelIndex><name></name><selectedRelationships>failure</selectedRelationships><source><groupId>defb04c4-c15c-3a07-0000-000000000000</groupId><id>8a0e37da-acd2-3d72-0000-000000000000</id><type>PROCESSOR</type></source><zIndex>0</zIndex></connections><connections><id>45e5403f-99f7-3ddf-0000-000000000000</id><parentGroupId>defb04c4-c15c-3a07-0000-000000000000</parentGroupId><backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold><backPressureObjectThreshold>10000</backPressureObjectThreshold><destination><groupId>defb04c4-c15c-3a07-0000-000000000000</groupId><id>9f8f32f7-130c-35bd-0000-000000000000</id><type>PROCESSOR</type></destination><flowFileExpiration>0 sec</flowFileExpiration><labelIndex>1</labelIndex><name></name><selectedRelationships>success</selectedRelationships><source><groupId>defb04c4-c15c-3a07-0000-000000000000</groupId><id>8a0e37da-acd2-3d72-0000-000000000000</id><type>PROCESSOR</type></source><zIndex>0</zIndex></connections><controllerServices><id>88b0195a-34b2-34f0-0000-000000000000</id><parentGroupId>defb04c4-c15c-3a07-0000-000000000000</parentGroupId><bundle><artifact>nifi-record-serialization-services-nar</artifact><group>org.apache.nifi</group><version>1.6.0</version></bundle><comments></comments><descriptors><entry><key>Schema Write Strategy</key><value><name>Schema Write Strategy</name></value></entry><entry><key>schema-access-strategy</key><value><name>schema-access-strategy</name></value></entry><entry><key>schema-registry</key><value><identifiesControllerService>org.apache.nifi.schemaregistry.services.SchemaRegistry</identifiesControllerService><name>schema-registry</name></value></entry><entry><key>schema-name</key><value><name>schema-name</name></value></entry><entry><key>schema-version</key><value><name>schema-version</name></value></entry><entry><key>schema-branch</key><value><name>schema-branch</name></value></entry><entry><key>schema-text</key><value><name>schema-text</name></value></entry><entry><key>Date Format</key><value><name>Date Format</name></value></entry><entry><key>Time Format</key><value><name>Time Format</name></value></entry><entry><key>Timestamp Format</key><value><name>Timestamp Format</name></value></entry><entry><key>Pretty Print JSON</key><value><name>Pretty Print JSON</name></value></entry><entry><key>suppress-nulls</key><value><name>suppress-nulls</name></value></entry></descriptors><name>JsonRecordSetWriter</name><persistsState>false</persistsState><properties><entry><key>Schema Write Strategy</key><value>no-schema</value></entry><entry><key>schema-access-strategy</key></entry><entry><key>schema-registry</key></entry><entry><key>schema-name</key></entry><entry><key>schema-version</key></entry><entry><key>schema-branch</key></entry><entry><key>schema-text</key></entry><entry><key>Date Format</key></entry><entry><key>Time Format</key></entry><entry><key>Timestamp Format</key></entry><entry><key>Pretty Print JSON</key></entry><entry><key>suppress-nulls</key></entry></properties><state>ENABLED</state><type>org.apache.nifi.json.JsonRecordSetWriter</type></controllerServices><controllerServices><id>c3e80a29-498b-36d4-0000-000000000000</id><parentGroupId>defb04c4-c15c-3a07-0000-000000000000</parentGroupId><bundle><artifact>nifi-record-serialization-services-nar</artifact><group>org.apache.nifi</group><version>1.6.0</version></bundle><comments></comments><descriptors><entry><key>schema-access-strategy</key><value><name>schema-access-strategy</name></value></entry><entry><key>schema-registry</key><value><identifiesControllerService>org.apache.nifi.schemaregistry.services.SchemaRegistry</identifiesControllerService><name>schema-registry</name></value></entry><entry><key>schema-name</key><value><name>schema-name</name></value></entry><entry><key>schema-version</key><value><name>schema-version</name></value></entry><entry><key>schema-branch</key><value><name>schema-branch</name></value></entry><entry><key>schema-text</key><value><name>schema-text</name></value></entry><entry><key>csv-reader-csv-parser</key><value><name>csv-reader-csv-parser</name></value></entry><entry><key>Date Format</key><value><name>Date Format</name></value></entry><entry><key>Time Format</key><value><name>Time Format</name></value></entry><entry><key>Timestamp Format</key><value><name>Timestamp Format</name></value></entry><entry><key>CSV Format</key><value><name>CSV Format</name></value></entry><entry><key>Value Separator</key><value><name>Value Separator</name></value></entry><entry><key>Skip Header Line</key><value><name>Skip Header Line</name></value></entry><entry><key>ignore-csv-header</key><value><name>ignore-csv-header</name></value></entry><entry><key>Quote Character</key><value><name>Quote Character</name></value></entry><entry><key>Escape Character</key><value><name>Escape Character</name></value></entry><entry><key>Comment Marker</key><value><name>Comment Marker</name></value></entry><entry><key>Null String</key><value><name>Null String</name></value></entry><entry><key>Trim Fields</key><value><name>Trim Fields</name></value></entry><entry><key>csvutils-character-set</key><value><name>csvutils-character-set</name></value></entry></descriptors><name>CSVReader</name><persistsState>false</persistsState><properties><entry><key>schema-access-strategy</key></entry><entry><key>schema-registry</key></entry><entry><key>schema-name</key></entry><entry><key>schema-version</key></entry><entry><key>schema-branch</key></entry><entry><key>schema-text</key></entry><entry><key>csv-reader-csv-parser</key></entry><entry><key>Date Format</key></entry><entry><key>Time Format</key></entry><entry><key>Timestamp Format</key></entry><entry><key>CSV Format</key></entry><entry><key>Value …
我有json喜欢
{
"error_message_1": "missing_field_1",
"error_message_2": "missing_field_2",
"error_message_3": "missing_field_3"
}{
"payloads":[
{
"error_message_1": "missing_field_1",
"error_message_2": "missing_field_2"
"error_message_3": "missing_field_3"
}],
"timestamp":"$timestmap",
"source":"nifi"
}
请让我知道这个jolt脚本.