C++ serialization of static array of struct using Cereal

Question

I am using Cereal library for serialization of structured data, to send them over MPI and later on process them on GPU using CUDA. Due to the latter, I cannot use std::vector, because CUDA has problems with dynamic objects. Therefore I am using static arrays of structures.

GOAL

To do binary serialization & deserialization on objects with nested static array of objects.

OBJECTS TO BE SERIALIZED

//==========================================================//
//                  ELEMENT DEFINITION                      //
//==========================================================//
class element{
public:
    //==========================//
    //          CONTENT         //
    //==========================//
    int grid_id;
    int NN_id;
    int debug;

    //==========================//
    //      SERIALIZATION       //
    //==========================//
    // function required by cereal library
    template<class Archive>
    void serialize(Archive & ar){
        ar( grid_id );
        ar( NN_id );
        ar( debug );
    }
};

//==========================================================//
//                      CONTAINER DEFINITION                //
//==========================================================//
class GPU_in_grid : public Managed{
public:
    //==========================//
    //          CONTENT         //
    //==========================//
    int gpu_id;
    element element[GRID_SIZE];        <------ static array of structures
    int debug;


    //==========================//
    //      SERIALIZATION       //
    //==========================//
    template<class Archive>
    void serialize(Archive & ar){
        ar( gpu_id );
        ar( debug );
        ar( element );

    }
};

SERIALIZATION FUNCTIONS

// cereal serialization
#include <cereal/archives/binary.hpp>
#include <cereal/archives/portable_binary.hpp>
//#include <cereal/archives/xml.hpp>

template<typename data_type>
int Process::serialize(data_type * data, char *serial_data, int *size){
    // serialize the data
    std::ostringstream oss(std::ios::binary);
    cereal::BinaryOutputArchive ar(oss);
    //cereal::XMLOutputArchive ar(std::cout);
    ar(*data);
    std::string s=oss.str();
    *size = s.length();
    strncpy(serial_data, s.c_str(), s.length());
    std::cout << "buffer["<< s.length() << "] >> " << s << std::endl;
    s.clear();
    return _SUCCESS_;
};

template<typename data_type>
int Process::deserialize(data_type * data, char *serial_data, int size){
    // create temporary buffer to store the received data
    char * buf=new char[size];
    strncpy(buf, serial_data, size);
    std::istringstream iss(std::string(serial_data, size), std::ios::binary);
    cereal::BinaryInputArchive arin(iss);
    arin(*data);
    // clean buffer
    delete[] buf;
    return _SUCCESS_;
};

DEBUGGING OUTPUT

cpu_mem BEFORE serialization
    cpu_mem.debug = -1
    cpu_mem.gpu_id = -5
    cpu_mem.element[0].NN_id = 1  
    cpu_mem.element[0].debug = 2  
buffer[248] >> ��������           <------ binary format
cpu_mem AFTER deserialization
    cpu_mem.debug = -1            <----- CORRECT
    cpu_mem.gpu_id = -5           <----- CORRECT
    cpu_mem.element[0].NN_id = 0  <----- INCORRECT
    cpu_mem.element[0].debug = 0  <----- INCORRECT

If I play around a bit with the streams, I can print out the serialized object as XML, to check whether the serialization is successful.

<?xml version="1.0" encoding="utf-8"?>
<cereal>
    <value0>
        <gpu_id>-5</gpu_id>               <----- CORRECT
        <debug>-1</debug>                 <----- CORRECT
        <element>
            <value0>
                <grid_id>0</grid_id>
                <NN_id>1</NN_id>          <----- CORRECT
                <debug>2</debug>          <----- CORRECT
            </value0>
            <value1>
                <grid_id>32522</grid_id>
                <NN_id>2</NN_id>
                <debug>4612412</debug>
            </value1>
        </element>
    </value0>
</cereal>

PROBLEM

The above outputs shows, that deserialization correctly recognizes variables in the container (class GPU_in_grid) but cannot deserialize the lower levels of my structure, namely static array of structures -> element[].

martin · Accepted Answer · 2017-11-30 16:12:47Z

1

For some unknown reason to me, the strncpy() malfunctions. The content of the string differs from the content of the array, EVEN if I keep eye on '\0', which need to be appended at the end of the char array, shown here http://www.cplusplus.com/reference/string/string/copy/ .

I have ended up using std::string.copy() instead, which performs as expected, as long as you add '\0' at the end of the char array.

the above code works then just fine.

answered Nov 30, 2017 at 16:12

martin

1007 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

C++ serialization of static array of struct using Cereal

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related