Unspecified byte lengths in Python

Question

I'm writing a client for a P2P application at the minute and the spec for the protocol says that the header for each packet should have each field with a particular byte length like so:

Version: 1 Byte 
Type: 1 Byte
Length: 2 Bytes
And then the data

I've got the way of packing and unpacking the header fields (I think) like this:

packed = struct.pack('cch' , '1' , '1' , 26)

This constructs a header for a packet with a data length of 26, but when it comes to unpacking the data I'm unsure how to go about getting the rest of the data afterwards. To unpack we need to know the size of all the fields, unless I'm missing something? I guess to pack the data I'd use a format indicator 'cch26s' meaning:

1 Byte char
1 Byte char
2 Byte short
26 Byte char array

But how do I unpack the data when I don't know how much data will be included in the packet first?

alexis · Accepted Answer · 2012-03-05 12:54:29Z

3

The way you're describing the protocol, you should unpack the first four bytes first, and extract Length (a 16-bit int). This tells you how many bytes to unpack in a second step.

version, type, length = struct.unpack("cch", packed[:4])
content, = struct.unpack("%ds" % length, packed[4:])

This is if everything checks out. unpack() requires that the packed buffer contain exactly as much data as you unpack. Also, check whether the 4 header bytes are included in the length count.

edited Mar 5, 2012 at 12:54

answered Mar 5, 2012 at 12:48

alexis

50.4k18 gold badges108 silver badges173 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

unutbu · Accepted Answer · 2012-03-05 11:51:48Z

You can surmise the number of characters to unpack by inspecting len(data).

Here is a helper function which does this for you:

def unpack(fmt, astr):
    """
    Return struct.unpack(fmt, astr) with the optional single * in fmt replaced with
    the appropriate number, given the length of astr.
    """
    # http://stackoverflow.com/a/7867892/190597
    try:
        return struct.unpack(fmt, astr)
    except struct.error:
        flen = struct.calcsize(fmt.replace('*', ''))
        alen = len(astr)
        idx = fmt.find('*')
        before_char = fmt[idx-1]
        n = (alen-flen)/struct.calcsize(before_char)+1
        fmt = ''.join((fmt[:idx-1], str(n), before_char, fmt[idx+1:]))
        return struct.unpack(fmt, astr)

You can use it like this:

unpack('cchs*', data)

Collectives™ on Stack Overflow

Unspecified byte lengths in Python

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related