YAML预处理器/宏处理器

Jus*_*tin 5 preprocessor yaml

是否有一种简单的方法可以将预处理器/宏处理器与YAML文件一起使用?(即我正在考虑C预处理器的内容)?

我们有很多描述各种数据结构的平面文本文件.它们目前采用我们自己的内部格式,并使用内部解析器进行读取.我想切换到YAML文件,以利用各种预先存在的库进行读写.

但是我们的文件是分层的,因为我们将主文件"包含"到子文件中,并且使用变量替换生成新的数据结构.

作为一个玩具示例,我想要的东西如下:

country_master.yaml

name: $COUNTRY$
file: C:\data\$COUNTRY$
Run Code Online (Sandbox Code Playgroud)

UK_country.yaml

#define $COUNTRY$ UK
#include <country_master.yaml>
Run Code Online (Sandbox Code Playgroud)

USA_country.yaml

#define $COUNTRY$ USA
#include <country_master.yaml>
Run Code Online (Sandbox Code Playgroud)

然后在预处理之后我们得到类似的东西:

name: USA
file: C:\data\USA
Run Code Online (Sandbox Code Playgroud)

C预处理器不能使用YAML注释中使用的#字符.另外,理想情况下我们希望有一个由预处理器扩展的循环,所以在上面的例子中我们创建了一个带有循环的UK和USA(我不相信你可以循环cpp).

有任何想法吗?

小智 6

# Yamp - YAML Macro-Processor
# https://github.com/birchb1024/yamp

# in master.yaml
defmacro:
  name: country
  args: [$COUNTRY$]
  value:
    name: $COUNTRY$
    file: C:\data\{{$COUNTRY$}}
---
# in some file
- include: [master.yaml]

# Call with wherever needed:
{ country: USA }
Run Code Online (Sandbox Code Playgroud)


Ant*_*hon 1

您试图在 YAML 的字符串表示级别上进行更改,但我认为您不应该这样做。YAML 可以加载对象,并且这些对象可以通过挂钩到解析器来影响后来加载的元素。这样您就可以用数据替换完整的节点,更改标量内的值等。

\n\n

假设您有这个 YAML 文件main.yml

\n\n
- !YAMLPreProcessor\n  verbose: \'3\'\n  escape: \xe2\x99\xa6\n- \xe2\x99\xa6replace(verbose)\n- abcd\n- \xe2\x99\xa6include(xyz.yml)\n- xyz\n
Run Code Online (Sandbox Code Playgroud)\n\n

xyz.yml就是:

\n\n
k: 9\nl: 8\nm: [7. 6]   # can be either\n
Run Code Online (Sandbox Code Playgroud)\n\n

并且您有\xe2\x99\xa6特殊字符(它可以是任何东西,只要特殊的 YAMLPreProcessor 值与操作关键字(replaceinclude)的开头匹配即可。您希望它是往返的(加载到内存中的数据中,然后转储到以下 YAML :

\n\n
- !YAMLPreProcessor\n  verbose: \'3\'\n  escape: \xe2\x99\xa6\n- \'3\'\n- abcd\n- k: 9\n  l: 8\n  m: [7. 6] # can be either\n- xyz\n
Run Code Online (Sandbox Code Playgroud)\n\n

您可以通过重载为每个标量和适当的类调用的标量构造函数来做到这一点YAMLPreProcessor

\n\n
# coding: utf-8\n\nfrom __future__ import print_function\n\nimport ruamel.yaml as yaml\n\ndef construct_scalar(loader, node):\n    self = getattr(loader, \'_yaml_preprocessor\', None)\n    if self and self.d.get(\'escape\'):\n        if node.value and node.value.startswith(self.d[\'escape\']):\n            key_word, rest = node.value[1:].split(\'(\', 1)\n            args, rest = rest.split(\')\', 1)\n            if key_word == \'replace\':\n                res = u\'\'\n                for arg in args.split(\',\'):\n                    res += str(self.d[arg])\n                node.value = res + rest\n            elif key_word == \'include\':\n                inc_yml = yaml.load(\n                    open(args),\n                    Loader=yaml.RoundTripLoader\n                )\n                # this needs ruamel.yaml>=0.9.6\n                return inc_yml\n            else:\n                print(\'keyword not found:\', key_word)\n    ret_val = loader._org_construct_scalar(node)\n    # print(\'ret_val\', type(ret_val), ret_val)\n    return ret_val\n\nclass YAMLPreProcessor:\n    def __init__(self, escape=None, verbose=0):\n        self.d = dict(escape=escape, verbose=verbose)\n\n    def __repr__(self):\n        return "YAMLPreProcessor({escape!r}, {verbose})".format(**self.d)\n\n    @staticmethod\n    def __yaml_out__(dumper, self):\n        return dumper.represent_mapping(\'!YAMLPreProcessor\', self.d)\n\n    @staticmethod\n    def __yaml_in__(loader, data):\n        from ruamel.yaml.comments import CommentedMap\n        result = YAMLPreProcessor()\n        loader._yaml_preprocessor = result\n        z = dict()\n        loader.construct_mapping(data, z)\n        result.d = z\n        yield result\n\n    def __delete__(self):\n        loader._yaml_preprocessor = None\n\n\n\ndef construct_yaml_str(self, node):\n    value = self.construct_scalar(node)\n    if isinstance(value, ScalarString):\n        return value\n    if PY3:\n        return value\n    try:\n        return value.encode(\'ascii\')\n    except AttributeError:\n        # in case you replace the node dynamically e.g. with a dict\n        return value\n    except UnicodeEncodeError:\n        return value\n\n\nloader = yaml.RoundTripLoader\n\nloader.add_constructor(\'!YAMLPreProcessor\', YAMLPreProcessor.__yaml_in__)\nloader._org_construct_scalar = loader.construct_scalar\nloader.construct_scalar = construct_scalar\n\ndata_from_yaml = yaml.load(open(\'main.yml\'), Loader=loader)\n\n#print (\'out\', data_from_yaml)\n\ndumper = yaml.RoundTripDumper\n# need to be able to represent \'!YAMLPreProcessor\'\n# but you can of course also remove the first element\n# from data_from_yaml if you don\'t want the preprocessor in your output\ndumper.add_representer(YAMLPreProcessor, YAMLPreProcessor.__yaml_out__)\n\nprint(yaml.dump(data_from_yaml, Dumper=dumper, allow_unicode=True))\n
Run Code Online (Sandbox Code Playgroud)\n\n

如果construct_scalar返回非字符串对象,上面需要最新版本的ruamel.yaml(0.9.6)作为旧版本\nchoke。

\n\n

请注意,带有键的行后面的注释位置m是相对于行的开头的,并且在示例中\n没有对xyz.yml插入文件的节点的缩进级别进行补偿。

\n