0

I have a program that serializes data with Protobuf in Java, by writing binary data in a byte[] Array and then saving it in ".txt" file. I am receiving that data on the C++ side in a stringstream. Now I want to parse that binary data with C++, but the Protobuf-Parsing-Method "parseFromString()" doesn't work! The fields from my Test Message are not set. I wrote a little test for that and I can show you some code:

Java Serialization

  byte[] s = test.build().toByteArray(); //This is serialized to "C:\test.txt" as binary

C++ Parsing:

Test t1; // My Protobuf Message
std::ifstream myFile("C:\\test.txt");
std::string s;
myFile >> s;

t1.ParseFromString(s);
std::cout << "Decoded: " << t2.s() << std::endl; // Check if parsing was correct

But it just returns: " Decoded: ", as if t2 was empty, but it shouldn't be! How can you parse binary data in C++?

Improve this question

2 Answers 2

Reset to default
5

Your problem is probably here:

myFile >> s;

The >> operator reads a text string, delimited by whitespace. An encoded protobuf is not text. Probably, ParseFromString() is returning false to indicate that it couldn't parse the data, because it is incomplete.

What you want to do is read the whole file. The easiest way to do this in your case is to use ParseFromIstream(&myFile). (And make sure to check that it returns true!)

(Another option would be to check the file size, create an array of that size, use myFile.read(array, size), and then ParseFromArray(array, size), but that's a lot more work that will do the same thing.)

Note that you probably should not use .txt as a file extension for a protobuf as the file does not contain text.

Also note that, confusingly, in C++ you can put binary (non-text) data in an std::string -- it's just a byte container -- but in Java you cannot put binary data in a String. So the C++ ParseFromString() and SerializeAsString() deal in binary data, whereas the Java toString() actually returns a textual representation of the message (meant for debugging purposes) which is not what you want to transmit.

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

Perfect answer, that was the problem! Thank you very much for your time :)
0

The problem is that you are mixing API calls. You're serializing with toByteArray API call and deserializing with ParseFromString API call.

You can use one of these pairs, but you can't mix them:

  • ParseFromString and SerializeToString pair
  • SerializeToArray and ParseFromArray pair

I'm programming C++ and python. I attached below example from my code. I think you should be able to understand how it is working from this example. If something is ambiguous please leave me a comment.

Here is my python serialization:

socket.send(request.SerializeToString())

and C++ deserialization:

// Deserialize request.
if (request.ParseFromString(protobuf_request) == false) {
    throw exception();
}

Here is my C++ serialization:

// Serialize response.
assert(response.SerializeToString(&protobuf_response) == true);

and python deserialization:

response.ParseFromString(str(message))

EDIT:

I think you should use:

  • void writeTo(OutputStream output); in Java
  • bool ParseFromIstream(istream* input); in C++
Improve this answer

6 Comments

Hi, Thank you for your complete answer. That should be the problem for sure. I tried the ParseFromArray() API call with a string as data source in C++, but it still isn't able to parse the raw data. Which API parsing call in C++ might be suitable to load the java-generated binary?
[Serialization in Java] void writeTo(OutputStream output);
[Deserialization in C++] bool ParseFromIstream(istream* input);
This answer is wrong. In C++, ParseFromString() and ParseFromArray() parse exactly the same format; the only difference is the former expects the buffer to be contained in an std::string (which in C++ is allowed to contain binary bytes, not just text) while the latter expects a pointer and a size. You can actually call ParseFromArray(str.data(), str.size()) and it will do the right thing!
I see. First I will double check it and then I will update my answer accordingly. Thanks!
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.