使用 serde 解/序列化 yaml 或 json 时如何解构枚举?

Mar*_*cin 1 serialization yaml rust deserialization serde

我有一段serde代码可以实现我想要的功能,但我不喜欢它的实现方式。我正在寻求帮助以了解如何改进它。

\n

操场

\n
use std::any::Any;\n\ntrait Model: std::fmt::Debug + Any {\n    fn as_any(&self) -> &dyn Any;\n}\n\nimpl Model for Generator {\n    fn as_any(&self) -> &dyn Any {\n        self\n    }\n}\nimpl Model for Connector {\n    fn as_any(&self) -> &dyn Any {\n        self\n    }\n}\n\n#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]\npub struct Generator {\n    id: String,\n    #[serde(rename = "sourceID")]\n    source_id: String,\n}\n\n#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]\npub struct Connector {\n    id: String,\n    #[serde(rename = "sourceID")]\n    source_id: String,\n}\n\n#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]\n#[serde(tag = "type")]\nenum AllModels {\n    Generator(Generator),\n    Connector(Connector),\n}\n\nfn main() {\n    let data = r#"\n- sourceID: "generator-01"\n  id: "connector-01"\n  type: "Generator"\n- sourceID: "geneiator-01"\n  type: "Connector"\n  id: "connector-01"\n"#;\n    let p: Vec<Box<dyn Model>> = serde_yaml::from_str::<Vec<AllModels>>(&data)\n        .unwrap()\n        .into_iter()\n        .collect();\n    println!("{:?}", p);\n    let l = serde_yaml::to_string(&p.into_iter().collect::<Vec<AllModels>>());\n    println!("{:?}", l);\n}\n\nimpl std::iter::FromIterator<AllModels> for Vec<Box<dyn Model>> {\n    fn from_iter<I: IntoIterator<Item = AllModels>>(iter: I) -> Self {\n        let mut v: Vec<Box<dyn Model>> = Vec::new();\n        for i in iter {\n            match i {\n                AllModels::Generator(d) => {\n                    v.push(Box::new(d));\n                }\n                AllModels::Connector(d) => {\n                    v.push(Box::new(d));\n                }\n            }\n        }\n        v\n    }\n}\n\nimpl std::iter::FromIterator<std::boxed::Box<dyn Model>> for std::vec::Vec<AllModels> {\n    fn from_iter<I: IntoIterator<Item = Box<dyn Model>>>(iter: I) -> Self {\n        let mut v: Vec<AllModels> = Vec::new();\n        for i in iter {\n            if let Some(model) = i.as_any().downcast_ref::<Generator>() {\n                v.push(AllModels::Generator(model.clone()));\n            } else if let Some(model) = i.as_any().downcast_ref::<Connector>() {\n                v.push(AllModels::Connector(model.clone()));\n            }\n        }\n        v\n    }\n}\n
Run Code Online (Sandbox Code Playgroud)\n

我想要实现的是将 yaml 反/序列化为多个结构之一,根据它type解析的 yaml 中的字段值动态选择应该反序列化到哪个结构。例如

\n
- id: Foo\n  source: Bar\n  type: Connector\n
Run Code Online (Sandbox Code Playgroud)\n

应该被解析为struct Connector

\n

我想我可以使用枚举表示来处理这个问题,但是,它会产生不需要的副作用 - 默认情况下遵循 yaml:

\n
- id: Foo\n  source: Bar\n  type: Connector\n- id: Foo\n  source: Bar\n  type: Generator\n
Run Code Online (Sandbox Code Playgroud)\n

将被解析为:

\n
[Connector(Connector{...}), Generator(Generator{...})]\n
Run Code Online (Sandbox Code Playgroud)\n

所以我的结构被封装在枚举变体中。为了“解开它”,我想我可以实现FromIterator<AllModels> for Vec<Box<dyn Model>>,多亏了它和类型转换/强制(不确定哪个是正确的词),输出更改为:

\n
[Connector{...}, Generator{...}]\n
Run Code Online (Sandbox Code Playgroud)\n

到目前为止,一切都很好。

\n

我在使用此解决方案时遇到的两个问题是:

\n
    \n
  1. 代码重复 - 对于每个新结构(连接器,生成器,...)我必须更新enum AllModelsmatch武装内部FromIterator实现 - 后者是最困扰我的。我可能可以用宏来做到这一点,但我还没有学会如何编写它们,在这样做之前,我想探索其他可能的解决方案
  2. \n
  3. 额外的迭代 - 为了从 转换Vec<enum variant>Vec<struct>我需要执行以下操作:let p: Vec<Box<dyn Model>> = serde_yaml::from_str::<Vec<AllModels>>(&data).unwrap().into_iter().collect();。我希望在没有额外迭代的情况下进行转换
  4. \n
\n

我考虑了一些选项,但我不知道如何实现它们......

\n

A. 来自/进入文档的serde 容器属性

\n

#[serde(from = "FromType")]- 我认为它的工作方式是将我的枚举变量直接强制转换为所需的结构,没有额外的迭代,也没有代码重复。但是,我未能实现它 - Playground

\n

当我尝试添加from属性时

\n
#[derive(Debug, Clone, Serialize, Deserialize)]\n#[serde(tag = "type")]\nenum AllModels {\n  Gene(Generator),\n  Connector(Connector),\n}\n
Run Code Online (Sandbox Code Playgroud)\n

编译器会对我大喊大叫

\n
 \xe2\x9d\xaf cargo r\n   Compiling serdeissue v0.1.0 (/sandbox/serdeissue)\nerror[E0277]: the trait bound `Box<dyn Model>: From<AllModels>` is not satisfied\n  --> src/main.rs:21:24\n   |\n21 | #[derive(Debug, Clone, Serialize, Deserialize)]\n   |                        ^^^^^^^^^ the trait `From<AllModels>` is not implemented for `Box<dyn Model>`\n   |\n   = help: the following implementations were found:\n             <Box<(dyn StdError + \'a)> as From<E>>\n             <Box<(dyn StdError + \'static)> as From<&str>>\n             <Box<(dyn StdError + \'static)> as From<Cow<\'a, str>>>\n             <Box<(dyn StdError + \'static)> as From<std::string::String>>\n           and 22 others\n   = note: required because of the requirements on the impl of `Into<Box<dyn Model>>` for `AllModels`\n   = note: required by `into`\n   = note: this error originates in a derive macro (in Nightly builds, run with -Z macro-backtrace for more info)\n\nerror[E0277]: the trait bound `dyn Model: Serialize` is not satisfied\n   --> src/main.rs:21:24\n    |\n21  | #[derive(Debug, Clone, Serialize, Deserialize)]\n    |                        ^^^^^^^^^ the trait `Serialize` is not implemented for `dyn Model`\n    |\n   ::: /home/marcin/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.123/src/ser/mod.rs:247:18\n    |\n247 |     fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>\n    |                  - required by this bound in `serialize`\n    |\n    = note: required because of the requirements on the impl of `Serialize` for `Box<dyn Model>`\n    = note: this error originates in a derive macro (in Nightly builds, run with -Z macro-backtrace for more info)\n\nerror[E0277]: the trait bound `dyn Model: Deserialize<\'_>` is not satisfied\n   --> src/main.rs:21:35\n    |\n21  | #[derive(Debug, Clone, Serialize, Deserialize)]\n    |                                   ^^^^^^^^^^^ the trait `Deserialize<\'_>` is not implemented for `dyn Model`\n    |\n   ::: /home/marcin/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.123/src/de/mod.rs:539:12\n    |\n539 |         D: Deserializer<\'de>;\n    |            ----------------- required by this bound in `_::_serde::Deserialize::deserialize`\n    |\n    = note: required because of the requirements on the impl of `Deserialize<\'_>` for `Box<dyn Model>`\n    = note: this error originates in a derive macro (in Nightly builds, run with -Z macro-backtrace for more info)\n\nerror[E0277]: the trait bound `AllModels: From<Box<dyn Model>>` is not satisfied\n   --> src/main.rs:21:35\n    |\n21  | #[derive(Debug, Clone, Serialize, Deserialize)]\n    |                                   ^^^^^^^^^^^ the trait `From<Box<dyn Model>>` is not implemented for `AllModels`\n    |\n   ::: /home/marcin/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/convert/mod.rs:372:1\n    |\n372 | pub trait From<T>: Sized {\n    | ------------------------ required by this bound in `From`\n    |\n    = note: this error originates in a derive macro (in Nightly builds, run with -Z macro-backtrace for more info)\n
Run Code Online (Sandbox Code Playgroud)\n

我的攻击角度如下:使用错误消息复制 Pasteroni-dummy-implementoni 缺失的特征边界:

\n
impl From<AllModels> for Box<dyn Model> {\n    fn from(am: AllModels) -> Self {\n        Box::new(Generator{id:String::from("arst"),source_id:String::from("arst")})\n    }\n}\nimpl Serialize for dyn Model {\n    fn serialize(&self) -> Self {\n        Box::new(Generator{id:String::from("arst"),source_id:String::from("arst")})\n    }\n}\n\nimpl Deserialize<\'_> for dyn Model {}\nimpl From<Box<dyn Model>> for AllModels {\n    fn from(dm: Box<dyn Model>) -> Self {\n        AllModels::Gene(Generator{id:String::from("arst"),source_id:String::from("arst")})\n    }\n}\n
Run Code Online (Sandbox Code Playgroud)\n

但随后会发生这种情况:

\n
 \xe2\x9d\xaf cargo r\n   Compiling serdeissue v0.1.0 (/sandbox/serdeissue)\nerror[E0277]: the size for values of type `(dyn Model + \'static)` cannot be known at compilation time\n   --> src/main.rs:75:6\n    |\n75  | impl Deserialize<\'_> for dyn Model {}\n    |      ^^^^^^^^^^^^^^^ doesn\'t have a size known at compile-time\n    |\n   ::: /home/marcin/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.123/src/de/mod.rs:530:29\n    |\n530 | pub trait Deserialize<\'de>: Sized {\n    |                             ----- required by this bound in `Deserialize`\n    |\n    = help: the trait `Sized` is not implemented for `(dyn Model + \'static)`\n
Run Code Online (Sandbox Code Playgroud)\n

对我来说游戏就结束了

\n

B.擦除-serde

\n

这似乎是完成这项工作的正确工具,但我在实现它时再次遇到了问题(难怪 - 我不知道我在做什么:)

\n
use erased_serde::{Deserializer,  Serializer, serialize_trait_object};\nuse serde::{Serialize, Deserialize};\n\n\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct Generator {\n  id: String,\n  source: String,\n}\n\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct Connector {\n  id: String,\n  source: String,\n}\n\ntrait Model: std::fmt::Debug + erased_serde::Serialize {}\nerased_serde::serialize_trait_object!(Model);\nimpl Model for Generator {}\nimpl Model for Connector {}\n\n#[derive(Debug, Clone, Serialize, Deserialize)]\n#[serde(tag = "type", into = "Box<dyn Model>", from = "Box<dyn Model>")]\nenum AllModels {\n  Generator(Generator),\n  Connector(Connector),\n}\n\nfn main() {\n  let data = r#"\n- source: "generator-01"\n  id: "g-01"\n  type: "Generator"\n- source: "connector-01"\n  type: "Connector"\n  id: "c-01"\n"#;\n  let p: Vec<Box<dyn Model>> = serde_yaml::from_str(&data).unwrap();\n  println!("{:?}", p);\n}\n\n\nimpl From<AllModels> for Box<dyn Model> {\n    fn from(am: AllModels) -> Self {\n        Box::new(Generator{id:String::from("arst"),source_id:String::from("arst")})\n    }\n}\n// impl Serialize for dyn Model {\n//     fn serialize(&self) -> Self {\n//         Box::new(Generator{id:String::from("arst"),source_id:String::from("arst")})\n//     }\n// }\nimpl Deserialize<\'_> for dyn Model {}\nimpl From<Box<dyn Model>> for AllModels {\n    fn from(dm: Box<dyn Model>) -> Self {\n        AllModels::Generator(Generator{id:String::from("arst"),source_id:String::from("arst")})\n    }\n}\n\n// impl std::convert::From<AllModels> for Box<dyn Model> {\n//     fn from(m: AllModels) -> Self {\n//        Box::new(Generator {source_id: String::from("i"), id: String::from("r")})\n//     }\n//     }\n\n
Run Code Online (Sandbox Code Playgroud)\n

编译时出现此错误:

\n
 \xe2\x9d\xaf cargo r\n   Compiling serdeissue v0.1.0 (/sandbox/serdeissue)\nwarning: unused imports: `Deserializer`, `Serializer`, `serialize_trait_object`\n --> src/main.rs:1:20\n  |\n1 | use erased_serde::{Deserializer,  Serializer, serialize_trait_object};\n  |                    ^^^^^^^^^^^^   ^^^^^^^^^^  ^^^^^^^^^^^^^^^^^^^^^^\n  |\n  = note: `#[warn(unused_imports)]` on by default\n\nerror[E0277]: the size for values of type `(dyn Model + \'static)` cannot be known at compilation time\n   --> src/main.rs:76:6\n    |\n76  | impl Deserialize<\'_> for dyn Model {}\n    |      ^^^^^^^^^^^^^^^ doesn\'t have a size known at compile-time\n    |\n   ::: /home/marcin/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.123/src/de/mod.rs:530:29\n    |\n530 | pub trait Deserialize<\'de>: Sized {\n    |                             ----- required by this bound in `Deserialize`\n    |\n    = help: the trait `Sized` is not implemented for `(dyn Model + \'static)`\n
Run Code Online (Sandbox Code Playgroud)\n

我认为erased_serde可以帮助解决这个问题,也许确实如此,但我不知道如何实现它。

\n
\n

遗憾的是我无法使用typetag板条箱,因为它不支持wasm我需要的编译目标。我不考虑#[serde(serialize_with = "path")]对每个枚举变体使用,因为它使我的问题#1比目前更糟糕。

\n

我也知道这个问题How to Implement `serde::Serialize` for a boxed Trait object? 但是@dtolnay提供的代码无法编译,我不知道如何修复它

\n
\xe2\x9d\xaf cargo r\n   Compiling serdeissue v0.1.0 (/sandbox/serdeissue)\nerror[E0603]: module `export` is private\n   --> src/main.rs:168:10\n    |\n168 | #[derive(Serialize)]\n    |          ^^^^^^^^^ private module\n    |\nnote: the module `export` is defined here\n   --> /home/marcin/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.123/src/lib.rs:275:5\n    |\n275 | use self::__private as export;\n    |     ^^^^^^^^^^^^^^^^^^^^^^^^^\n\nerror[E0603]: module `export` is private\n   --> src/main.rs:173:10\n    |\n173 | #[derive(Serialize)]\n    |          ^^^^^^^^^ private module\n    |\nnote: the module `export` is defined here\n   --> /home/marcin/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.123/src/lib.rs:275:5\n    |\n275 | use self::__private as export;\n    |     ^^^^^^^^^^^^^^^^^^^^^^^^^\n\nerror[E0603]: module `export` is private\n   --> src/main.rs:179:10\n    |\n179 | #[derive(Serialize)]\n    |          ^^^^^^^^^ private module\n    |\nnote: the module `export` is defined here\n   --> /home/marcin/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.123/src/lib.rs:275:5\n    |\n275 | use self::__private as export;\n    |     ^^^^^^^^^^^^^^^^^^^^^^^^^\n\nerror[E0603]: module `export` is private\n   --> src/main.rs:184:10\n    |\n184 | #[derive(Serialize)]\n    |          ^^^^^^^^^ private module\n    |\nnote: the module `export` is defined here\n   --> /home/marcin/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.123/src/lib.rs:275:5\n    |\n275 | use self::__private as export;\n    |     ^^^^^^^^^^^^^^^^^^^^^^^^^\n\nwarning: trait objects without an explicit `dyn` are deprecated\n   --> src/main.rs:176:22\n    |\n176 |     widgets: Vec<Box<WidgetTrait>>,\n    |                      ^^^^^^^^^^^ help: use `dyn`: `dyn WidgetTrait`\n    |\n    = note: `#[warn(bare_trait_objects)]` on by default\n
Run Code Online (Sandbox Code Playgroud)\n

看起来我正在寻找的功能正在等待在这里实现: https: //github.com/serde-rs/serde/issues/1402

\n

还有这个问题https://github.com/serde-rs/serde/issues/1350建议手动解串器实现Playground。游乐场代码表明这可以帮助解决我的问题#2“额外迭代”,但是代码实现中仍然存在一些重复,因此我仍在寻找更好的答案。

\n
\n

编辑:我也在考虑Enum 或 Trait object,无法弄清楚评估我是否需要其中之一的正确方法是什么。

\n

Mar*_*cin 5

我找到了一个令我满意的解决方案:

  • 光滑的
  • 没有代码重复
  • 没有动态调度,没有额外的迭代
  • 用途enumserde(tag="...")
    • typejson/yaml 中的字段可能会乱序(不一定是第一个)
    • type序列化回 json/yaml 时包含字段

诀窍是使用enum_dispatch.

use serde::{Serialize, Deserialize};
use enum_dispatch::enum_dispatch;


#[derive(Debug, Clone, Serialize, Deserialize)]
struct Generator {
    source_id: String,
    id: String,
}

#[derive(Debug, Clone, Serialize, Deserialize)]
struct Gate {
    source_id: String,
    id: String,
}

#[enum_dispatch]
trait Model {fn time_advance(self, a:i32,b:i32) -> i32;} 
impl Model for Generator { fn time_advance(self,a:i32,b:i32) -> i32 {a+b} }
impl Model for Gate { fn time_advance(self,a:i32,b:i32) -> i32 {a+b} }

#[enum_dispatch(Model)]
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(tag = "type")]
enum ModelTypes {
    Generator(Generator),
    Gate(Gate),
}

fn main() {
    let s = r#"
- source_id: "generator-01"
  id: "connector-01"
  type: "Generator"
- source_id: "geneiator-01"
  type: "Gate"
  id: "connector-01"
"#;
    let data: Vec<ModelTypes> = serde_yaml::from_str(s).unwrap();
    println!("{:?}", serde_yaml::to_string(&data));
    for d in data {
        println!("{:?}", d.time_advance(4, 2));
    }
}
Run Code Online (Sandbox Code Playgroud)
[package]
name = "enum_unpack"
version = "0.1.0"
authors = ["---"]
edition = "2018"

[dependencies]
serde = { version = "1.0.124", features = ["derive"] }
serde_yaml = "0.8.1"
enum_dispatch = "0.3.5"
Run Code Online (Sandbox Code Playgroud)

输出:

[package]
name = "enum_unpack"
version = "0.1.0"
authors = ["---"]
edition = "2018"

[dependencies]
serde = { version = "1.0.124", features = ["derive"] }
serde_yaml = "0.8.1"
enum_dispatch = "0.3.5"
Run Code Online (Sandbox Code Playgroud)