我是 GCP 数据流的新手。
我尝试从 GCP 云存储中将文本文件(一行 JSON 字符串)读取为 JSON 格式,然后根据特定字段的值将其拆分并输出到 GCP 云存储(作为 JSON 字符串文本文件)。
这是我的代码
但是,我在 GCP 数据流上遇到了一些错误:
Traceback (most recent call last):
File "main.py", line 169, in <module>
run()
File "main.py", line 163, in run
shard_name_template='')
File "C:\ProgramData\Miniconda3\lib\site-packages\apache_beam\pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "C:\ProgramData\Miniconda3\lib\site-packages\apache_beam\runners\dataflow\dataflow_runner.py", line 1346, in wait_until_finish
(self.state, getattr(self._runner, 'last_error_msg', None)), self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/dataflow_worker/batchworker.py", line 773, in run
self._load_main_session(self.local_staging_directory)
File "/usr/local/lib/python3.7/site-packages/dataflow_worker/batchworker.py", line 489, in …Run Code Online (Sandbox Code Playgroud) 我尝试找到一种在MS Teams的个人聊天中发送私人消息的方法。
我知道在“通道 > 连接器”中,我可以使用 Webhook 来实现此目的。不幸的是,我在个人聊天中找不到类似的内容。
我发现的唯一相关的是 MS Graph API,但这不是我想要的。
目前,我正在尝试使用电子邮件 API 作为替代方案
任何建议将不胜感激,谢谢。
基于官方文件
我尝试使用“PubSub Pull Subscription”触发器创建云函数
import base64
def hello_pubsub(event, context):
"""Triggered from a message on a Cloud Pub/Sub topic.
Args:
event (dict): Event payload.
context (google.cloud.functions.Context): Metadata for the event.
"""
print("This Function was triggered by messageId {} published at {}".format(context.event_id, context.timestamp))
if 'data' in event:
name = base64.b64decode(event['data']).decode('utf-8')
print('"{}" received!'.format(name))
if 'attributes' in event:
print(event['attributes'])
if '@type' in event:
print(event['@type'])
Run Code Online (Sandbox Code Playgroud)

然后我发现一篇文章说“云函数在调用时会发送ACK”,这与官方文档一致。
但是,当云函数处理完 PubSub 消息后,“Unacked message count”并没有减少(如上图所示)
因此,我在本地尝试google-cloud-pubsub
subscription_path = subscriber.subscription_path(PROJECT, SUBSCRIPTION)
response = subscriber.pull(subscription_path, max_messages=5)
for …Run Code Online (Sandbox Code Playgroud) python google-cloud-platform google-cloud-pubsub google-cloud-functions