小编use*_*549的帖子

分析node.js中的大型json日志文件

我有以下JSON文件:

sensorlogs.json
{"arr":[{"UTCTime":10000001,"s1":22,"s2":32,"s3":42,"s4":12},
{"UTCTime":10000002,"s1":23,"s2":33,"s4":13},
{"UTCTime":10000003,"s1":24,"s2":34,"s3":43,"s4":14},
{"UTCTime":10000005,"s1":26,"s2":36,"s3":44,"s4":16},
{"UTCTime":10000006,"s1":27,"s2":37,"s4":17},
{"UTCTime":10000004,"s1":25,"s2":35,"s4":15},
...
{"UTCTime":12345678,"s1":57,"s2":35,"s3":77,"s4":99}
]}
Run Code Online (Sandbox Code Playgroud)

传感器s1,s2,s3等都以不同的频率发送(注意s3每2秒发送一次,并且时间表可能出现故障).

我怎样才能实现像 -

Analyzing s1:
s = [[10000001, 22], [10000002, 23],.. [12345678,57]]
s1 had 2 missing entries
Analyzing s2:
s = [[10000001, 32], [10000002, 33],.. [12345678,35]]
s2 had 0 missing entries
Analyzing s3:
s = [[10000001, 42], [10000003, 43],.. [12345678,77]]
s3 had 0 missing entries
Analyzing s4:
s = [[10000001, 12], [10000003, 13],.. [12345678,99]]
s4 had 1 missing entries
Run Code Online (Sandbox Code Playgroud)

sensorlogs.json是16 GB.

可以根据连续UTC时间戳的差异找到缺少的条目.每个传感器以已知频率传输.

由于内存限制,我无法使用多个大型数组进行分析,因此我必须在同一个JSON日志文件上进行多次传递,并且只使用单个大型数组进行分析.

到目前为止我所拥有的是 -

var result = []; …
Run Code Online (Sandbox Code Playgroud)

javascript json node.js

7
推荐指数
1
解决办法
239
查看次数

标签 统计

javascript ×1

json ×1

node.js ×1