那些遇到过的问题

如何调试boost archive异常输入流错误

Bha*_*kar 2 c++ serialization boost

我正在尝试序列化以下类型的对象

std::unordered_map<std::vector<Card>, std::unordered_map<InfosetHistory, Node>>& nodeMap

Card是一个结构体，InfosetHistory并且Node是使用其他一些结构体作为成员变量的类。我已经serialize为所有需要它的类创建了函数。例如，这是一个Card：

struct Card {
    int rank;
    int suit;
    ...

    template<class Archive>
    void serialize(Archive& ar, const unsigned int version) {
        ar & rank;
        ar & suit;
    }
};
Run Code Online (Sandbox Code Playgroud)

我将nodeMap序列化如下：

std::ofstream file("Strategy" + std::to_string(iteration) + ".bin");
boost::archive::binary_oarchive archive(file);
archive << nodeMap;
file.close();
Run Code Online (Sandbox Code Playgroud)

并像这样单独反序列化（当前选择反序列化“Strategy0.bin”）：

std::ifstream ifs("Strategy0.bin");
std::unordered_map<std::vector<Card>, std::unordered_map<InfosetHistory, Node>> nodeMap;
if (ifs.good()) {
    boost::archive::binary_iarchive ia(ifs);
    ia >> nodeMap;
}
Run Code Online (Sandbox Code Playgroud)

当我运行程序来创建和序列化 nodeMap 时，我始终能够毫无问题地进行序列化。创建了相应的 .bin 文件，它们的大小似乎适合我希望它们存储的数据。

然而，当我运行程序来反序列化nodeMap时，如果nodeMap不是那么大，我不会遇到问题，但如果它很大，我会收到以下错误：

terminate called after throwing an instance of 'boost::archive::archive_exception'
  what():  input stream error
Run Code Online (Sandbox Code Playgroud)

我认为这实际上并不是因为 nodeMap 太大，而是因为代码创建的条目有可能以某种方式引起问题，并且添加的条目越多，遇到问题的可能性就越大。我在 Boost 文档中读到，这种错误可能是由于未初始化的数据造成的。我不相信我有未初始化的数据，但我不知道如何确保这一点。

一般来说，我不确定如何调试此类问题。任何帮助，将不胜感激。

注意：我非常努力地创建一个最小的可重现示例，但我创建的所有示例都没有产生问题。只有当我在程序中创建此类对象并添加数千个条目时，我才会遇到此类问题。

编辑 @sehe 要求更多代码。以下是与序列化对象相关的类和结构的所有相关部分：

https://pastebin.com/xPE8h8a3

请注意，这些类和结构位于单独的文件中。InfosetHistory和DrawAction被声明于InfosetHistory.h、Node被声明于Node.h、 且Card被声明于GRUtil.h。

@sehe 还提到我没有提到上面序列化的类型。正在序列化的类型是对我尝试序列化的对象的引用：std::unordered_map<std::vector<Card>, std::unordered_map<InfosetHistory, Node>>& nodeMap。

编辑2 我已经设法使用下面提供的代码@sehe 创建了一个最小的可重现示例。使用我的代码，我创建了一个我知道会产生反序列化错误的 NodeMap，并将所有数据打印到名为“StrategyWritten0.txt”的文本文件中。在这个可重现的示例中，我输入该文本文件中的所有数据来创建 NodeMap，序列化生成的 NodeMap 中的所有数据，然后尝试反序列化 NodeMap。我得到以下输出：

Successfully created Node Map
Serialized Node Map
terminate called after throwing an instance of 'boost::archive::archive_exception'
  what():  input stream error
Run Code Online (Sandbox Code Playgroud)

这是文件：

https://drive.google.com/file/d/1y4FLgi7f-XWJ-igRq_tItK-pXFDEUbhg/view?usp=sharing

这是代码：

https://pastebin.com/FebQqssx

seh*_*ehe 5

更新

经过几天的努力（在虚拟机上安装 MingW 并调试具体细节），我将范围缩小到关联容器的第 25 个元素之后发生的一些特殊情况。不久之后，我灵机一动：

在此输入图像描述

是的。Windows 的行尾毁掉了一些人美好的几天。再次。

我总是在我的存档文件上写上std::ios::binary标志，但在这种特殊情况下，我没有发现它们丢失了。

添加它们将解决问题：

void writeBinaryStrategy(std::string const& fname, NodeMap const& nodeMap) {
    std::ofstream f(fname, std::ios::binary);
    boost::archive::binary_oarchive archive(f);
    archive << nodeMap;
}

NodeMap readBinaryStrategy(std::string const& fname) {
    NodeMap nodeMap;
    std::ifstream ifs(fname, std::ios::binary);

    boost::archive::binary_iarchive ia(ifs);
    ia >> nodeMap;

    return nodeMap;
}
Run Code Online (Sandbox Code Playgroud)

在深入研究细节的过程中，我想出了一个严格的往返测试器应用程序。该代码在 Coliru 上~unlive~，所以我将这些文件放在一个 Gist 中：

  • 文件test.cpp

    #include <iostream>
#include <boost/archive/binary_oarchive.hpp>
#include <boost/archive/binary_iarchive.hpp>
#include <boost/serialization/serialization.hpp>
#include <boost/serialization/unordered_map.hpp>
#include <boost/serialization/map.hpp>
#include <boost/serialization/vector.hpp>
#include <boost/serialization/array.hpp>
#include <boost/functional.hpp>
#include <boost/container_hash/hash.hpp>

using boost::hash_value;

struct Card {
    int rank, suit;

    Card(int rank = 0, int suit = 0) : rank(rank), suit(suit) {}

    template<class Archive>
    void serialize(Archive& ar, unsigned /*version*/) {
        ar & rank;
        ar & suit;
    }

    friend size_t hash_value(Card const& c) {
        auto v = hash_value(c.rank);
        boost::hash_combine(v, hash_value(c.suit));
        return v;
    }

    auto tied() const { return std::tie(rank, suit); }
    bool operator<(Card const& rhs) const { return tied() < rhs.tied(); }
    bool operator==(Card const& rhs) const { return tied() == rhs.tied(); }
};

struct DrawAction {
    bool fromDeck;
    Card card;

    explicit DrawAction(bool fromDeck = false, Card card = {})
            : fromDeck(fromDeck), card(card) {}

    template<class Archive>
    void serialize(Archive& ar, unsigned /*version*/) {
        ar & fromDeck;
        ar & card;
    }

    friend size_t hash_value(DrawAction const& da) {
        auto v = hash_value(da.fromDeck);
        boost::hash_combine(v, hash_value(da.card));
        return v;
    }

    auto tied() const { return std::tie(fromDeck, card); }
    bool operator<(DrawAction const& rhs) const { return tied() < rhs.tied(); }
    bool operator==(DrawAction const& rhs) const { return tied() == rhs.tied(); }
};

using Cards = std::vector<Card>;
using Hand  = std::array<Card, 10>;
using Draws = std::vector<DrawAction>;

class InfosetHistory {
public:
    Card initialDiscardPile;
    Hand initialHand;
    Draws playerDrawActions;
    Cards playerDiscardActions;
    Draws opponentDrawActions;
    Cards opponentDiscardActions;

    InfosetHistory(
            Card initialDiscardPile = {},
            Hand hand = {},
            Draws playerDrawActions = {},
            Cards playerDiscardActions = {},
            Draws opponentDrawActions = {},
            Cards opponentDiscardActions = {}
    ) : initialDiscardPile(initialDiscardPile),
        initialHand(std::move(hand)),
        playerDrawActions(std::move(playerDrawActions)),
        playerDiscardActions(std::move(playerDiscardActions)),
        opponentDrawActions(std::move(opponentDrawActions)),
        opponentDiscardActions(std::move(opponentDiscardActions)) {}

    template<class Archive>
    void serialize(Archive& ar, const unsigned int /*version*/) {
        ar & initialDiscardPile & initialHand
        & playerDrawActions & playerDiscardActions
        & opponentDrawActions & opponentDiscardActions;
    }

    friend size_t hash_value(InfosetHistory const& ish) {
        auto v = hash_value(ish.initialDiscardPile);

        auto combine = [&v](auto& range) { boost::hash_range(v, begin(range), end(range)); };
        combine(ish.initialHand);
        combine(ish.playerDrawActions);
        combine(ish.playerDiscardActions);
        combine(ish.opponentDrawActions);
        combine(ish.opponentDiscardActions);
        return v;
    }

    auto tied() const { return std::tie(initialDiscardPile, initialHand,
            playerDrawActions, playerDiscardActions, opponentDrawActions,
            opponentDiscardActions); }

    bool operator<(InfosetHistory const& rhs) const { return tied() < rhs.tied(); }
    bool operator==(InfosetHistory const& rhs) const { return tied() == rhs.tied(); }
};

class Node {
public:
    Cards allowedActions;
    unsigned int NUM_ACTIONS{};
    std::vector<double> regretSum;
    std::vector<double> strategySum;
    unsigned char phase{};

    explicit Node(std::vector<Card> allowedActions = {},
                  unsigned int NUM_ACTIONS = 0,
                  std::vector<double> regretSum = {},
                  std::vector<double> strategySum = {},
                  unsigned char phase = 0
    ) : allowedActions(std::move(allowedActions)),
        NUM_ACTIONS(NUM_ACTIONS),
        regretSum(std::move(regretSum)),
        strategySum(std::move(strategySum)),
        phase(phase) {}

    template<class Archive>
    void serialize(Archive& ar, unsigned /*version*/) {
        ar & allowedActions
        & NUM_ACTIONS
        & regretSum & strategySum & phase;
    }

    auto tied() const { return std::tie(allowedActions, NUM_ACTIONS, regretSum, strategySum, phase); }
    bool operator<(Node const& rhs) const { return tied() < rhs.tied(); }
    bool operator==(Node const& rhs) const { return tied() == rhs.tied(); }
};

#include <map>
#include <fstream>

#if defined(ORDERED_MAP)
    template<typename K, typename V>
    using htable = std::map<K, V>;

    template <typename K, typename V>
    static inline bool check_insert(htable<K, V>& t, K k, V v) {
        return t.emplace(std::move(k), std::move(v)).second;
    }
#elif defined(UNORDERED_MAP)
    template<typename K, typename V>
    using htable = std::unordered_map<K, V, boost::hash<K> >;

    template <typename K, typename V>
    static inline bool check_insert(htable<K, V>& t, K k, V v) {
        return t.emplace(std::move(k), std::move(v)).second;
    }
#elif defined(INPUT_PRESERVING) // retain exact input order
    #include <boost/multi_index_container.hpp>
    #include <boost/multi_index/sequenced_index.hpp>
    #include <boost/multi_index/hashed_index.hpp>
    #include <boost/multi_index/member.hpp>

    namespace bmi = boost::multi_index;

    template<typename K, typename V, typename P = std::pair<K, V> >
    using htable = bmi::multi_index_container<
        P,
        bmi::indexed_by<
            bmi::sequenced<>,
            bmi::hashed_unique<bmi::member<P, K, &P::first>, boost::hash<K> >
        >
    >;
    template <typename K, typename V>
    static inline bool check_insert(htable<K, V>& t, K k, V v) {
        return t.insert(t.end(), std::make_pair(std::move(k), std::move(v))).second;
    }
#endif

using NodeMap = htable<Hand, htable<InfosetHistory, Node>>;

NodeMap readTextStrategy(std::string const& fname);
NodeMap readBinaryStrategy(std::string const& fname);

void writeTextStrategy(std::string const& fname, NodeMap const& nodeMap);
void writeBinaryStrategy(std::string const& fname, NodeMap const& nodeMap);

int main() {
    auto const original = readTextStrategy("StrategyWritten0.txt");

    NodeMap bin = original, txt;

    for (int i = 1; i<5; ++i) {
        auto const fname = "roundtrip" + std::to_string(i);

        writeBinaryStrategy(fname + ".bin", bin);
        writeTextStrategy(fname + ".txt", bin);
        bin = readBinaryStrategy(fname + ".bin");
        txt = readTextStrategy(fname + ".txt");

        std::cout << "text roundtrip " << i << " is " << (txt == original?"equal":"different") << "\n";
        std::cout << "bin  roundtrip " << i << " is " << (bin == original?"equal":"different") << "\n";
    }
}

void writeBinaryStrategy(std::string const& fname, NodeMap const& nodeMap) {
    std::ofstream f(fname, std::ios::binary);
    boost::archive::binary_oarchive archive(f);
    archive << nodeMap;
}

NodeMap readBinaryStrategy(std::string const& fname) {
    NodeMap nodeMap;
    std::ifstream ifs(fname, std::ios::binary);

    boost::archive::binary_iarchive ia(ifs);
    ia >> nodeMap;

    return nodeMap;
}

#include <iomanip>
#include <boost/lexical_cast.hpp> // full precision see /sf/answers/3365971661/
namespace TextSerialization {
#if defined(__MINGW32__) || defined(WIN32)
    static constexpr char const* CRLF = "\n";
#else
    static constexpr char const* CRLF = "\r\n";
#endif


    static inline std::ostream& operator<<(std::ostream& os, Card const& c) {
        return os << c.rank << CRLF << c.suit;
    }

    static inline std::ostream& operator<<(std::ostream& os, DrawAction const& da) {
        return os << da.fromDeck << CRLF << da.card;
    }

    template <typename T, size_t N>
    static inline std::ostream& operator<<(std::ostream& os, std::array<T, N> const& from) {
        auto n = N;
        for (auto& el : from)
            os << el << (--n?CRLF:"");
        return os;
    }

    template <typename... T>
    static inline std::ostream& operator<<(std::ostream& os, std::vector<T...> const& from) {
        os << from.size();
        for (auto& el : from)
            os << CRLF << el;
        return os;
    }

    template <typename K, typename V>
    static inline std::ostream& operator<<(std::ostream& os, htable<K, V> const& from) {
        auto n = from.size();
        os << n;
        for (auto& [k,v] : from)
            os << CRLF << k << CRLF << v;
        return os;
    }

    static inline std::ostream& operator<<(std::ostream& os, InfosetHistory const& ish) {
        return os
            << ish.initialHand         << CRLF << ish.initialDiscardPile << CRLF
            << ish.playerDrawActions   << CRLF << ish.playerDiscardActions << CRLF
            << ish.opponentDrawActions << CRLF << ish.opponentDiscardActions;
    }

    static inline std::ostream& operator<<(std::ostream& os, Node const& n) {
        assert(n.NUM_ACTIONS == n.regretSum.size());
        assert(n.NUM_ACTIONS == n.strategySum.size());

        os << n.allowedActions << CRLF
           << n.NUM_ACTIONS << CRLF;
        for (auto& v: {n.regretSum, n.strategySum})
            for (auto& el: v)
                os << boost::lexical_cast<std::string>(el) << CRLF;
        return os << n.phase;
    }
}

namespace TextDeserialization {
    template <typename Cont>
    static inline void read_n(std::istream& is, size_t n, Cont& into) {
        while (n--)
            is >> *into.emplace(into.end());
    }

    static inline std::istream& operator>>(std::istream& is, Card& c) {
        return is >> c.rank >> c.suit;
    }

    static inline std::istream& operator>>(std::istream& is, DrawAction& da) {
        return is >> da.fromDeck >> da.card;
    }

    template <typename T, size_t N>
    static inline std::istream& operator>>(std::istream& is, std::array<T, N>& into) {
        for (auto& el : into)
            is >> el;
        return is;
    }

    template <typename... T>
    static inline std::istream& operator>>(std::istream& is, std::vector<T...>& into) {
        size_t n;
        is >> n;
        read_n(is, n, into);
        return is;
    }

    template <typename K, typename V>
    static inline std::istream& operator>>(std::istream& is, htable<K, V>& into) {
        size_t n;
        is >> n;
        K k; V v;
        while (n--) {
            if (is >> k >> v && !check_insert(into, std::move(k), std::move(v)))
                throw std::range_error("duplicate key");
        }
        return is;
    }

    static inline std::istream& operator>>(std::istream& is, InfosetHistory& ish) {
        return is
            >> ish.initialHand >> ish.initialDiscardPile
            >> ish.playerDrawActions >> ish.playerDiscardActions
            >> ish.opponentDrawActions >> ish.opponentDiscardActions;
    }

    static inline std::istream& operator>>(std::istream& is, Node& n) {
        is >> n.allowedActions;
        is >> n.NUM_ACTIONS;
        read_n(is, n.NUM_ACTIONS, n.regretSum);
        read_n(is, n.NUM_ACTIONS, n.strategySum);
        return is >> n.phase;
    }
}

void writeTextStrategy(std::string const& fname, NodeMap const& nodeMap) {
    using namespace TextSerialization;
    std::ofstream os(fname);
    os << nodeMap << CRLF;
}

NodeMap readTextStrategy(std::string const& fname) {
    using namespace TextDeserialization;
    std::ifstream is(fname);

    NodeMap nodeMap;
    is >> nodeMap;

    return nodeMap;
}
    Run Code Online (Sandbox Code Playgroud)

  • 文件CMakeLists.txt

    PROJECT(work)
CMAKE_MINIMUM_REQUIRED(VERSION 3.16)

SET(CMAKE_CXX_STANDARD 17)
SET(CMAKE_CXX_FLAGS "-g -O0 -L .")

LINK_DIRECTORIES("C:\\boost\\lib")
LINK_LIBRARIES(boost_serialization-mgw8-mt-d-x64-1_73)
INCLUDE_DIRECTORIES("C:\\boost\\include\\boost-1_73")

ADD_EXECUTABLE(ordered test.cpp)
TARGET_COMPILE_DEFINITIONS(ordered PRIVATE "-DORDERED_MAP")

ADD_EXECUTABLE(unordered test.cpp)
TARGET_COMPILE_DEFINITIONS(unordered PRIVATE "-DUNORDERED_MAP")

ADD_EXECUTABLE(multi-index test.cpp)
TARGET_COMPILE_DEFINITIONS(multi-index PRIVATE "-DINPUT_PRESERVING")
    Run Code Online (Sandbox Code Playgroud)

建造：

mkdir -p build && (cd build; cmake ..; cmake --build .)
Run Code Online (Sandbox Code Playgroud)

当使用保留输入顺序的数据结构（multi-index, with -DINPUT_PRESERVING）进行测试时：

./build/multi-index.exe
md5sum roundtrip*.txt StrategyWritten0.txt
Run Code Online (Sandbox Code Playgroud)

印刷

text roundtrip 1 is equal
bin  roundtrip 1 is equal
text roundtrip 2 is equal
bin  roundtrip 2 is equal
text roundtrip 3 is equal
bin  roundtrip 3 is equal
text roundtrip 4 is equal
bin  roundtrip 4 is equal
d62dd8fe217595f2e069eabf54de479a *roundtrip1.txt
d62dd8fe217595f2e069eabf54de479a *roundtrip2.txt
d62dd8fe217595f2e069eabf54de479a *roundtrip3.txt
d62dd8fe217595f2e069eabf54de479a *roundtrip4.txt
d62dd8fe217595f2e069eabf54de479a *StrategyWritten0.txt
Run Code Online (Sandbox Code Playgroud)

原答案

我将nodeMap序列化如下：

std::ofstream file("Strategy" + std::to_string(iteration) + ".bin");
boost::archive::binary_oarchive archive(file);
archive << nodeMap;
file.close();
Run Code Online (Sandbox Code Playgroud)

这会带来存档不完整的风险，这可能是您的问题。您在档案析构函数运行之前关闭该文件。更好的：

我将nodeMap序列化如下：

{ 
    std::ofstream file("Strategy" + std::to_string(iteration) + ".bin");
    boost::archive::binary_oarchive archive(file);
    archive << nodeMap;
}
Run Code Online (Sandbox Code Playgroud)

这样，存档的析构函数会在文件的析构函数之前运行（作为奖励，您不必手动关闭它）。

（析构函数以与构造相反的顺序运行）。

其他需要记住的事情

  • 对这个答案所付出的所有努力表示支持;)。 (2认同)

归档时间：

查看次数：

1066 次

最近记录：

5 年，3 月 前
Boost的lexical_cast从双精度到字符串精度 4
更多相关链接
相关归档
std :: stoi实际上是否可以安全使用？ 58
用于音频处理的任何类似OpenCV的C/C++库？ 50
模板专业化VS函数重载 50
带有符号和无符号操作数的'&' 28
将boost :: function降级为普通函数指针 21
为什么我无法在C#中序列化元组？ 18
C# - 如何xml反序列化对象本身？ 8
java项目中所有类的单元测试可序列化性 5
允许PHP解析大文件比调用MySQL数据库中的数据更快吗？ 3
关于shared_ptr,scoped_ptr和shared_array的一些问题 1
难疑归档
从数组创建ArrayList 3441
Vim清除上次搜索突出显示 1824
如何在不安装Ms Office的情况下在C#中创建Excel(.XLS和.XLSX)文件？ 1804
将存储过程的结果插入临时表 1526
Dockerfile中CMD和ENTRYPOINT有什么区别？ 1484
如何离开/退出/停用python virtualenv？ 1461
如何完全删除使用init创建的git存储库？ 1358
你如何存放未跟踪的文件？ 1287
是否有"以前的兄弟"CSS选择器？ 1253
"正确"的JSON日期格式 1071