3

I got a cstring, originating from a call from gzread. I know the data is blocks, and each block is consisting of an unsigned int, char, int and unsigned short int.

So I was wondering what the standard way of splitting this cstring into the appropriate variables is.

Say the first 4 bytes, is a unsigned int, the next byte is char, the next 4 bytes is signed int, and the last 2 bytes are unsigned short int.

//Some pseudocode below which would work
char buf[11];
unsigned int a;
char b;
int c;
unsigned short int d;

I guess I could memcpy, with appropriate offsets.

memcpy(&a, buf, sizeof(unsigned int));
memcpy(&b, buf+4, sizeof(char));
memcpy(&c, buf+5, sizeof(int));
memcpy(&d, buf+9, sizeof(unsigned short int));

Or is it better to use some bitoperators? Like shifting and masking.

Or would it be better to gzreading all 11 bytes directly into some struct, or is that even possible? Is the memory layout of a struct fixed, and will this work with gzread?

Improve this question

4 Answers 4

Reset to default
2

If you pack the struct (read up on __packed__ attribute), you can rely on the order and that the members are non-aligned. Hence, you could read into a struct directly. However, I'm not sure about the portability of this solution.

Otherwise, use pointer magic and casting like so:

char *buffer;
int a = *(reinterpret_cast<int*> (buffer))
unsigned short b = *(reinterpret_cast<unsigned short*> (buffer + sizeof(int)))
Improve this answer
Sign up to request clarification or add additional context in comments.

9 Comments

+1 for this one, although most modern hardware is little endian, probably apart from mainframes and specialized processors.
On systems which have strict alignment requirements, type-punning with pointers like this will fail if the buffer is not correctly aligned for access as the longer type.
-1 because it is simply wrong. Most processors have alignment requirements, which will cause the code to crash, and of course, processors vary greatly with regards to endianess, and in a few cases, even how they represent signed integers.
@ame Most modern processors are big-endian; the only exception I know is the Intel architecture (and Alpha, if it is still made). "Standard" Internet format is also big-endian. And of course, if you have to take mainframes into account, some aren't even 2's complement.
@James Kanze: Surely, it's not a way to design a portable binary format, but as serialization mean it's OK. If data are serialized and deserialized on the same machine, then endianess and alignments will be the same.
|
2

You need to make sure that the byte order of the file matches the processor architecture you're running your code on. If, for instance, the integers are written to file with most significant byte first and your processor uses least significant byte first order, you're getting garbage for results.

If you want to make your code portable from one architecture to another, you should wrap all read and write operations for integers behind macros or inline functions that manage the byte order for you depending on the target processor architecture.

Improve this answer

6 Comments

How exactly do you find out about the endianness of an input file? I always wondered if there was a way to do that without having any data apart from the file itself.
There's got to be a spec somewhere. If there isn't you're on your own. In that case some heuristics might help, though: you could assume smaller numbers like 1 are more common than 1073741824 (32 bit representation of 1b reversed).
@arne well, it'd have to be something encoded in the file. In the case of text files there's the optional BOM which can tell you the byte order of the file en.wikipedia.org/wiki/Byte_order_mark
OK, but it all depends on the creator of the file :(
@arne: You could require that the first to bytes of your file make up a certain integer-value. Then you read them out, and see if the value is correct. If not, you have to swap bytes.
|
1

It depends on how the input data is defined. If it's defined to be in host-endian order (that is, the endianness always matches the system on which your code is running), then the memcpy() you have shown is a good, portable method to use.

Alternatively, if the input data is defined to have a particular endianness, then the best portable solution is to load it one unsigned char at a time, using shifts and bitwise-or.

Improve this answer

Comments

1

You need a specification of the format before you can do anything. Is it text or binary (presumably binary from your description, but one never knows)? What is the representation used for signed values? What is the byte order? memcpy will only work if your machine architecture corresponds exactly to that of the input format—a rare case today, since almost all network formats are big-endian, and the most widespread architectures are little-endian. (Most formats and architectures today use 2's complement for negative values, so you can often "assume" compatiblity there. But there are exceptions.)

Given this, mathematical reconstruction of the value (using masking and shifting, or multiplications) is the only portable solution. Depending on the machine and the quality of the compiler, it could easily result in better performance as well.

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.