Client-side string encoding java

Question

My team and I have this nasty problem with parsing a string received from our server. The server is pretty simple socket stuff done in qt here is the sendData function:

void sendData(QTcpSocket *client,QString response){
QString text = response.toUtf8();
QByteArray block;
QDataStream out(&block, QIODevice::WriteOnly);
out << (quint32)0;
out << text;
out.device()->seek(0);
out << (quint32)(block.size() - sizeof(quint32));
try{
    client->write(block);
}
catch(...){...

The client is in Java and is also pretty standard socket stuff, here is where we are at now after trying many many different ways of decoding the response from the server:

Socket s;
try {
    s = new Socket(URL, 1987);

    PrintWriter output = new PrintWriter(s.getOutputStream(), true);
    InputStreamReader inp = new InputStreamReader(s.getInputStream(), Charset.forName("UTF-8"));
    BufferedReader rd = new BufferedReader( inp );

    String st;
    while ((st = rd.readLine()) != null){
        System.out.println(st);
    }...

If a connection is made with the server it sends a string "Send Handshake" with the size of the string in bytes sent before it as seen in the first block of code. This notifies the client that it should send authentication to the server. As of now the string we get from the server looks like this: �� S��e��n��d�� H��a��n��d��s��h��a��k��e

We have used tools such as string encode/decode tool to try and assess how the string is encoded but it fails on every configuration.

We are out of ideas as to what encoding this is, if any, or how to fix it. Any help would be much appreciated.

It look like an encoding issue. There is a "secret message" in the string received. In any case, use a tool like wireshark or tcpdump to look at the data-on-the-wire to eliminate the need to guess :) — user166390
– user166390, Commented Dec 13, 2011 at 21:41
Your code sample never assigns block. Is it supposed to have block=response.toUtf8()? — Thomas
– Thomas, Commented Dec 13, 2011 at 21:53

Arnold Spence · Accepted Answer · 2011-12-13 21:56:22Z

3

At a glance, the line where you convert the QString parameter to a Utf8 QByteArray and then back to a QString seems odd:

QString text = response.toUtf8();

When the QByteArray returned by toUtf8() is assigned to text, I think it is assumed that the QByteArray contains an Ascii (char*) buffer.

answered Dec 13, 2011 at 21:56

Arnold Spence

22.4k7 gold badges77 silver badges68 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

okin33 Over a year ago

Thanks for the help we solved the problem, it was indeed that line.

okin33 Over a year ago

When we convert the response to UTF8 we were trying to store it in a variable of type QString, which broke the encoding. Once you change the type of the text variable to a QByteArray it works. QByteArray text = response.toUtf8();

Dave Mateer · Accepted Answer · 2011-12-13 22:04:12Z

1

I'm pretty sure that QDataStream is intended to be used only within Qt. It provides a platform-independent way of serializing data that is then intended to be deserialized with another QDataStream somewhere else. As you noticed, it's including a lot of extra stuff besides your raw data, and that extra stuff is subject to change at the next Qt version. (This is why the documentation suggests including in your stream the version of QDataStream being used ... so it can use the correct deserialization logic.)

In other words, the extra stuff you are seeing is probably meta-data included by Qt and it is not guaranteed to be the same with the next Qt version. From the docs:

QDataStream's binary format has evolved since Qt 1.0, and is likely to continue evolving to reflect changes done in Qt. When inputting or outputting complex types, it's very important to make sure that the same version of the stream (version()) is used for reading and writing.

If you are going to another language, this isn't practical to use. If it is just text you are passing, use a well-known transport mechanism (JSON, XML, ASCII text, UTF-8, etc.) and bypass the QDataStream altogether.

answered Dec 13, 2011 at 22:04

Dave Mateer

18k15 gold badges102 silver badges153 bronze badges

1 Comment

okin33 Over a year ago

Yes you are probably right about that we do send the version number along with the string but we removed it while debugging for simplicity. We will try use it to parse the junk/meta data out front of the string.

Collectives™ on Stack Overflow

Client-side string encoding java

2 Answers 2

2 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related