veg*_*ego 0 python apache-beam
最近在学习apache beam,发现了一些像这样的python代码:
lines = p | 'read' >> ReadFromText(known_args.input)
# Count the occurrences of each word.
def count_ones(word_ones):
(word, ones) = word_ones
return (word, sum(ones))
counts = (lines
| 'split' >> (beam.ParDo(WordExtractingDoFn())
.with_output_types(unicode))
| 'pair_with_one' >> beam.Map(lambda x: (x, 1))
| 'group' >> beam.GroupByKey()
| 'count' >> beam.Map(count_ones))
Run Code Online (Sandbox Code Playgroud)
来自: https: //github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/wordcount.py#L92
|python中and的语法和用法是什么>>?
默认情况下|代表逻辑或按位OR运算符,以及>>右移,但幸运的是你可以在 Python 中重载运算符。因此,为了对|and进行自定义定义>>,,您只需在类__or__和中重载以下两个 dunder(magic) 方法__rshift__:
class A():
def __or__(self):
pass
def __rshift__(self):
pass
Run Code Online (Sandbox Code Playgroud)
我建议您阅读有关Python 数据模型的更多信息。
现在查看Beam Python SDK,在类__or__中进行了重载PTransform:
def __or__(self, right):
"""Used to compose PTransforms, e.g., ptransform1 | ptransform2."""
if isinstance(right, PTransform):
return _ChainedPTransform(self, right)
return NotImplemented
Run Code Online (Sandbox Code Playgroud)
| 归档时间: |
|
| 查看次数: |
1493 次 |
| 最近记录: |