9

I hope this question is appropriate for stackoverflow... What is the difference between storing raw data bytes (8 bits) in a std::string rather than storing them in std::vector<char>. I'm reading binary data from a file and storing those raw bytes in a std::string. This works well, there are no problems or issues with doing this. My program works as expected. However, other programmers prefer the std::vector<char> approach and suggest I stop using std::string as it's unsafe for raw bytes. So I'm wondering why might it be unsafe to use std::string to hold raw data bytes? I know std::string is most often used to store ASCII text, but a byte is a byte, so I don't understand the preference of the std::vector<char>.

Thanks for any advice!

Improve this question
2
  • 3
    It used to be that std::string was not guaranteed to provide contiguous storage, which matters if you do things like &s[0] to obtain a non-const pointer to data. But this is no longer true as of C++11. Commented Mar 8, 2012 at 23:57
  • 1
    possible duplicate of vector <unsigned char> vs string for binary data Commented Mar 9, 2012 at 1:39

4 Answers 4

Reset to default
16

The problem is not really whether it works or it doesn't. The problem is that it is utterly confusing for the next guy reading your code. std::string is meant for displaying text. Anybody reading your code will expect that. You'll declare your intent much better with a std::vector<char>.

It increases your WTF/min in code reviews.

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

I've never thought of it that way. Good point. I think of std::string as a container holding bytes. They may be ASCII bytes, but are not required to be. In my mind, std::string is std::bytes, but it's good to know that others think differently. I can see how this would be confusing.
For more points of view, when I think "byte", I think uint8_t. For the most part, I only use char when I'm actually holding character data, when I'm using buffers (e.g. using new char[] to allocate the memory in which I'm going to construct an object), or to play nice with some established API that uses char. I would generally prefer to have have a std::vector<uint8_t> to store raw byte data.
@Hurkyl uint8_t is not guaranteed to be one byte long. char is.
@zenith: It would be interesting to know the loophole that lets that work; I know CHAR_BIT has to be at least 8, and sizes of objects have to be in multiples of a byte. The path to uint8_t not existing is clear; how do you arrange for it to exist and with sizeof(uint8_t) > 1?
6

In C++03, using std::string to store an array of byte data was not a good idea. By the standard, std::string did not have to store data contiguously. C++11 fixed that so that it's data does have to be contiguous.

So it would not be functional to do this in C++03. Not unless you have personally vetted your C++ standard library implementation of std::string to ensure that it is contiguous.

Either way, I would suggest vector<char>. Generally, when you see string, you expect it to be a... string. You know, a sequence of characters in some form of encoding. A vector<char> makes it obvious that it isn't a string, but an array of bytes.

Improve this answer

1 Comment

Thank you. We use C++11 so the bytes are contiguous. My idea of std::string seems to be broader than most. I appreciate your opinion. It's good for me to understand why others find this confusing, even though it works.
3

Besides contiguous storage and code-clarity issues, I ran into some fairly insidious errors trying to use std::string to hold raw bytes.

Most of them centered around trying to convert a char array of bytes to std::string when interfacing with C libraries. For example:

std::string password = "pass\0word";
std::cout << password.length() << std::endl; // prints 4, not 9

Maybe you can fix that by specifying the length:

std::string password("pass\0word", 0, 9);
std::cout << password.length() << std::endl; // nope! still 4!

This is probably because the constructor expects to receive a C-string, not a byte array. There might be a better way, but I ended up with this:

std::string password("pass0word", 0, 9);
password[4] = '\0';
std::cout << password.length() << std::endl; // hurray! 9!

A little clunky. Thankfully I found this in unit testing, but I would have missed it if my test vectors didn't have null bytes. What makes this insidious is that the second approach above will work fine until the array contains a null byte.

So far std::vector<uint8_t> looks like a good option (thanks J.N. and Hurkyl):

char p[] = "pass\0word";
std::vector<uint8_t> password(p, p, p+9); // :)

Note: I haven't tried the iterator constructor with std::string, but this error is easy enough to make that it might be worth avoiding even the possibility.

Lessons learned:

  • Test byte-handling methods witih null byte-containing test vectors.
  • Be careful when (and I would say avoid) using std::string to hold raw bytes.
Improve this answer

1 Comment

Can you initialize a vector of char or uint8_t from a string literal in C++11?
0

Use std::string to store characters from text files. That way you can be rest assured that character encoding is done properly for you.

Use std::vector<std::byte> to store raw bytes from binary files. This will prevent unintended type conversions and ensure safe data manipulation. The exact representation of the data is crucial and intent is communicated.

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.