小编Eas*_*est的帖子

尝试捕获 s3 上的增量文件（上次处理后创建的文件）。为此，使用 boto3 过滤器迭代器通过查询 LastModified 值而不是返回所有文件列表并在客户端站点上进行过滤。

根据http://jmespath.org/ ?，以下查询有效并过滤以下 json respose；

filtered_iterator = page_iterator.search(
"Contents[?LastModified>='datetime.datetime(2016, 12, 27, 8, 5, 37, tzinfo=tzutc())'].Key")

for key_data in filtered_iterator:
print(key_data)

但是它失败了；

RuntimeError: xxxxxxx has failed: can't compare datetime.datetime to str

示例分页器响应；

{
"Contents": [{
    "LastModified": "datetime.datetime(2016, 12, 28, 8, 5, 31, tzinfo=tzutc())",
    "ETag": "1022dad2540da33c35aba123476a4622",
    "StorageClass": "STANDARD",
    "Key": "blah1/blah11/abc.json",
    "Owner": {
        "DisplayName": "App-AWS",
        "ID": "bfc77ae78cf43fd1b19f24f99998cb86d6fd8220dbfce0ce6a98776253646656"
    },
    "Size": 623
}, {
    "LastModified": "datetime.datetime(2016, 12, 28, 8, 5, 37, tzinfo=tzutc())",
    "ETag": "1022dad2540da33c35abacd376a44444",
    "StorageClass": "STANDARD",
    "Key": "blah2/blah22/xyz.json", …

在生成或使用数据时我没有看到任何失败,但是生产中存在大量重复消息.对于获得大约100k消息的小主题,有大约4k重复,虽然我说没有失败,并且最重要的是没有实现重试逻辑或设置配置值.

我还检查了那些重复消息的偏移值,每个消息都有不同的值,告诉我问题出在生产者身上.

任何帮助将受到高度赞赏

尝试通过Java lib与Google Cloud Bigtable交互,并在成功创建连接对象后,发送"list"命令,我收到以下command.run()方法错误:

command.run(connection, argsList.subList(1, argsList.size()));

特别:

Jul 06, 2015 1:12:15 PM io.grpc.transport.netty.ProtocolNegotiators$AbstractBufferingHandler fail
SEVERE: Transport failed during protocol negotiation
io.netty.channel.ChannelPipelineException: io.grpc.transport.netty.ProtocolNegotiators$1$1.handlerAdded() has thrown an exception; removed.
at io.netty.channel.DefaultChannelPipeline.callHandlerAdded0(DefaultChannelPipeline.java:600)
at io.netty.channel.DefaultChannelPipeline.callHandlerAdded(DefaultChannelPipeline.java:582)
at io.netty.channel.DefaultChannelPipeline.addFirst0(DefaultChannelPipeline.java:124)
at io.netty.channel.DefaultChannelPipeline.addFirst(DefaultChannelPipeline.java:108)
at io.netty.channel.DefaultChannelPipeline.addFirst(DefaultChannelPipeline.java:291)
at io.netty.channel.DefaultChannelPipeline.addFirst(DefaultChannelPipeline.java:246)
at io.grpc.transport.netty.ProtocolNegotiators$AbstractBufferingHandler.channelRegistered(ProtocolNegotiators.java:233)
at io.netty.channel.ChannelHandlerInvokerUtil.invokeChannelRegisteredNow(ChannelHandlerInvokerUtil.java:32)
at io.netty.channel.DefaultChannelHandlerInvoker.invokeChannelRegistered(DefaultChannelHandlerInvoker.java:50)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRegistered(AbstractChannelHandlerContext.java:114)
at io.netty.channel.DefaultChannelPipeline.fireChannelRegistered(DefaultChannelPipeline.java:833)
at io.netty.channel.AbstractChannel$AbstractUnsafe.register0(AbstractChannel.java:487)
at io.netty.channel.AbstractChannel$AbstractUnsafe.access$100(AbstractChannel.java:401)
at io.netty.channel.AbstractChannel$AbstractUnsafe$1.run(AbstractChannel.java:461)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:322)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:356)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:703)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalStateException: NPN/ALPN extensions not installed  

跟随; https://cloud.google.com/bigtable/docs/samples-simple-cli

任何帮助将非常感谢!