Serializing classes into one defined object

Question

I want to design an application protocol, in which I define a class called message which has a fixed header size of 4 bytes, and variable payload part. To assign the payload to the message I want to do the following way:

 message msg;
 IPAddressv4 ipv4("127.0.0.1");
 IPAddressv6 ipv6("::1");
 Algorithm   alg(2);

 msg.version(1);
 msg.ack(1);

 msg << ipv4 & ipv6 & alg;

Where IPAddressv4, IPAddressv6 and Algorithm are defined object with basic data type. For example ipv4 is a uint32 and ipv6 is 2*unit64 and algorithm is uint8.

In this way the final message size is: Fixed Header Size "4" + 4 + 16 + 1 = 25 Bytes

So can anyone give me the general way how to implement this in c++, or is there anyway to do that in boost library?

"I want to do the following way" - and what would it /do/ then? It's pretty code. No clue what you expect to /happen/ — sehe
– sehe, Commented May 4, 2014 at 20:02
I want to copy the contents of the objects ipv4, ipv6 and alg into the payload part of the message, as a stream of bytes. You can consider the payload of the message as an archive. In my case The payload size after doing this serialization will be 21 Bytes. — IoT
– IoT, Commented May 4, 2014 at 20:05
Why << and &? Is there some reason why you need the operator precedence? And what part don't you get? The syntactic ---dury--- sugar to make that syntax work ("what have you tried") or the building of the actual stream? — sehe
– sehe, Commented May 4, 2014 at 20:07
You can use << or & to indicate serialization of the data type into the message payload. The reason for that is to make it easy for the final users who want to use my protocol in their programs. I tried to do some coding but I have problem in the way to do serialization of multiple objects in the same order. For example ipv4 should be copied first to the payload of the message, later ipv6 and finally alg. When the receiver receives this message he needs to do the opposite way to deserialize it msg >> ipv4 & ipv6 & alg; — IoT
– IoT, Commented May 4, 2014 at 20:13
mixing the operators seems recipe for confusion. I'll ignore it for now. — sehe
– sehe, Commented May 4, 2014 at 20:14

sehe · Accepted Answer · 2014-05-05 00:12:39Z

So, I reckon you wanted a class that acts like a stream but with binary formatting.

Here's my simplistic interface for such a class (simplified):

struct Message
{
    // serialize all of the inserted buffer content to the given stream
    friend std::ostream& operator <<(std::ostream& os, Message const& msg);

    // read all of the given stream into the message buffer for extraction
    friend std::istream& operator >>(std::istream& is, Message const& msg);

    // insert integral data into the buffer, with fixed endianness
    template <typename Int>
        friend Message& operator <<(Message& msg, Int v);

    template <typename Int, size_t N>
        friend Message& operator <<(Message& msg, Int const (&v)[N]);

    // extract integral data from the buffer, with fixed endianness
    template <typename Int>
        friend Message& operator >>(Message& msg, Int& v);

    template <typename Int, size_t N>
        friend Message& operator >>(Message& msg, Int (&v)[N]);

    // default construction
    Message() : buffer { "\xee\x33" }
    {
        buffer.unsetf(std::ios::skipws);
    }

private:
    std::stringstream mutable buffer;
};

I left out the arcane details dealing with SFINAE/meta-programming. The constructor is quick and dirty, and shows a simplistic way to have a fixed message header (ee33 in this case).

Now you can use it like this:

std::string serialize_message()
{
    uint8_t  ipv4[4] = { 127, 0, 0, 1 };
    uint64_t ipv6[2] = { 0xdeadbeef00004b1d, 0xcafed00dcafebabe };
    uint8_t  alg     = 2;

    Message msg;
    msg << ipv4 << ipv6 << alg;

    // stringyfy
    std::stringstream ss;
    ss << msg;
    return ss.str();
}

int main()
{
    std::string const rep = serialize_message();
    std::cout << rep << std::flush; // for demo (view in hex)

    // reconstruct
    Message roundtrip;
    std::istringstream(rep) >> roundtrip;

    uint8_t  ipv4[4] = { 0 };
    uint64_t ipv6[2] = { 0 };
    uint8_t  alg     = 0;
    roundtrip >> ipv4 >> ipv6 >> alg;

    assert(rep == boost::lexical_cast<std::string>(roundtrip));
}

Note that the lexical_cast just uses the stream insertion operator listed above.

I've implemented this using Boost Spirit's binary parsers and generators.

The mapping of input types to parser/generator types are in a detail namespace.

See it Live On Coliru. Output:

0000000: ee33 7f00 0001 dead beef 0000 4b1d cafe  .3..........K...
0000010: d00d cafe babe 02                        .......

Full Code

#define LIVE_DANGEROUSLY
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/karma.hpp>
#include <boost/integer.hpp>
#include <sstream>

namespace detail
{
    using namespace boost::spirit;
    // meta programming helpers
    template <int bits, typename endianness = struct big> struct int_traits;

    template <typename endianness> struct int_traits<8, endianness/*ignored*/> { 
        typedef uint8_t                  type;
        typedef qi::byte_type            parser;
        typedef karma::byte_type         generator;
    };

    template <> struct int_traits<16, struct big> {
        typedef uint16_t                 type;
        typedef qi::big_word_type        parser;
        typedef karma::big_word_type     generator;
    };

    template <> struct int_traits<32, struct big> {
        typedef uint32_t                 type;
        typedef qi::big_dword_type       parser;
        typedef karma::big_dword_type    generator;
    };

    template <> struct int_traits<64, struct big> {
        typedef uint64_t                 type;
        typedef qi::big_qword_type       parser;
        typedef karma::big_qword_type    generator;
    };

    template <> struct int_traits<16, struct little> {
        typedef uint16_t                 type;
        typedef qi::little_word_type     parser;
        typedef karma::little_word_type  generator;
    };

    template <> struct int_traits<32, struct little> {
        typedef uint32_t                 type;
        typedef qi::little_dword_type    parser;
        typedef karma::little_dword_type generator;
    };

    template <> struct int_traits<64, struct little> {
        typedef uint64_t                 type;
        typedef qi::little_qword_type    parser;
        typedef karma::little_qword_type generator;
    };
}

struct Message
{
    friend std::ostream& operator <<(std::ostream& os, Message const& msg) {
        msg.buffer.seekg(0, std::ios_base::beg);
        return os << msg.buffer.rdbuf();
    }

    friend std::istream& operator >>(std::istream& is, Message const& msg) {
        msg.buffer.clear(); // this will overwrite our header
        msg.buffer.str("");
        return is >> msg.buffer.rdbuf();
    }

    template <typename Int>
    friend typename boost::enable_if<typename boost::is_integral<Int>::type, Message&>::type
        operator <<(Message& msg, Int v)
        {
            msg.buffer.seekp(0, std::ios_base::end);
            msg.append_int(v);
            return msg;
        }

    template <typename Int, size_t N>
    friend typename boost::enable_if<typename boost::is_integral<Int>::type, Message&>::type
        operator <<(Message& msg, Int const (&v)[N])
        {
            msg.buffer.seekp(0, std::ios_base::end);
            msg.append_int(v);
            return msg;
        }

    template <typename Int>
    friend typename boost::enable_if<typename boost::is_integral<Int>::type, Message&>::type
        operator >>(Message& msg, Int& v)
        {
            msg.extract_int(v);
            return msg;
        }

    template <typename Int, size_t N>
    friend typename boost::enable_if<typename boost::is_integral<Int>::type, Message&>::type
        operator >>(Message& msg, Int (&v)[N])
        {
            msg.extract_ints(v);
            return msg;
        }

    Message() : buffer { "\xee\x33" }
    {
        buffer.unsetf(std::ios::skipws);
    }

  private:
    std::stringstream mutable buffer;

    template <typename Int, int bits=std::numeric_limits<Int>::digits + std::numeric_limits<Int>::is_signed>
    void append_int(Int v)
    {
        namespace karma = boost::spirit::karma;
        static const typename detail::int_traits<bits>::generator gen {};
        karma::generate(std::ostreambuf_iterator<char>(buffer), gen, static_cast<typename detail::int_traits<bits>::type>(v));
    }

    template <typename Int, int bits=std::numeric_limits<Int>::digits + std::numeric_limits<Int>::is_signed>
    void extract_int(Int& v)
    {
        namespace qi = boost::spirit::qi;
        static const typename detail::int_traits<bits>::parser p {};
        boost::spirit::istream_iterator f(buffer), l;
        qi::parse(f, l, p, v);
    }

    template <typename Int, size_t N, int bits=std::numeric_limits<Int>::digits + std::numeric_limits<Int>::is_signed>
    void append_int(Int const(&v)[N])
    {
#ifdef LIVE_DANGEROUSLY // optimize, but implementation defined cast
        namespace karma = boost::spirit::karma;
        static const typename detail::int_traits<bits>::generator gen {};
        auto data = reinterpret_cast<typename detail::int_traits<bits>::type const*>(&v[0]);
        karma::generate(std::ostreambuf_iterator<char>(buffer), +gen, boost::make_iterator_range(data,data+N));
#else
        for(auto& i:v)
            append_int(i);
#endif
    }

    template <typename Int, size_t N, int bits=std::numeric_limits<Int>::digits + std::numeric_limits<Int>::is_signed>
    void extract_ints(Int (&v)[N])
    {
        for(auto& i:v)
            extract_int(i);
    }
};

std::string serialize_message()
{
    uint8_t  ipv4[4] = { 127, 0, 0, 1 };
    uint64_t ipv6[2] = { 0xdeadbeef00004b1d, 0xcafed00dcafebabe };
    uint8_t  alg     = 2;

    Message msg;
    msg << ipv4 << ipv6 << alg;

    // stringyfy
    std::stringstream ss;
    ss << msg;
    return ss.str();
}

int main()
{
    std::string const rep = serialize_message();
    std::cout << rep << std::flush; // for demo (view in hex)

    // reconstruct
    Message roundtrip;
    std::istringstream(rep) >> roundtrip;

    uint8_t  ipv4[4] = { 0 };
    uint64_t ipv6[2] = { 0 };
    uint8_t  alg     = 0;
    roundtrip >> ipv4 >> ipv6 >> alg;

    assert(rep == boost::lexical_cast<std::string>(roundtrip));
}

PS. Switch the default endianness globally by editing line 11: little vs. big endian

Collectives™ on Stack Overflow

Serializing classes into one defined object

1 Answer 1

Full Code

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Full Code

1 Comment

Your Answer

Sign up or log in

Post as a guest