Java InputStream encoding/charset

Question

Running the following (example) code

import java.io.*;

public class test {
    public static void main(String[] args) throws Exception {
        byte[] buf = {-27};
        InputStream is = new ByteArrayInputStream(buf);
        BufferedReader r = new BufferedReader(
                new InputStreamReader(is, "ISO-8859-1"));
        String s = r.readLine();
        System.out.println("test.java:9 [byte] (char)" + (char)s.getBytes()[0] + 
                " (int)" + (int)s.getBytes()[0]);
        System.out.println("test.java:10 [char] (char)" + (char)s.charAt(0) + 
                " (int)" + (int)s.charAt(0));
        System.out.println("test.java:11 string below");
        System.out.println(s);
        System.out.println("test.java:13 string above");
    }
}

gives me this output

test.java:9 [byte] (char)? (int)63
test.java:10 [char] (char)? (int)229
test.java:11 string below
?
test.java:13 string above

How do I retain the correct byte value (-27) in the line-9 printout? And consequently receive the expected output of the System.out.println(s) command (å).

Jon Skeet · Accepted Answer · 2010-06-15 08:47:09Z

23

If you want to retain byte values, don't use a Reader at all, ideally. To represent arbitrary binary data in text and convert it back to binary data later, you should use base16 or base64 encoding.

However, to explain what's going on, when you call s.getBytes() that's using the default character encoding, which apparently doesn't include Unicode character U+00E5.

If you call s.getBytes("ISO-8859-1") everywhere instead of s.getBytes() I suspect you'll get back the right byte value... but relying on ISO-8859-1 for this is kinda dirty IMO.

answered Jun 15, 2010 at 8:47

Jon Skeet

1.5m893 gold badges9.3k silver badges9.3k bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Tobbe Over a year ago

s.getBytes("ISO-8859-1") did the trick, thank you. I was only using it to track down where the file contents I was reading changed in the path from reading the file to presenting the data to the user.

Jon Skeet Over a year ago

@Tobbe: Glad it helped. It would be better not to convert it into text at all though, in future. Unless it really is an ISO-8859-1 encoded text file, of course.

Matthew Flaschen · Accepted Answer · 2010-06-15 09:06:57Z

11

As noted, getBytes() (no-arguments) uses the Java platform default encoding, which may not be ISO-8859-1. Simply printing it should work, provided your terminal and the default encoding match and support the character. For instance, on my system, the terminal and default Java encoding are both UTF-8. The fact that you're seeing a '?' indicates that yours don't match or å is not supported.

If you want to manually encode to UTF-8 on your system, do:

String s = r.readLine();
byte[] utf8Bytes = s.getBytes("UTF-8");

It should give a byte array with {-61, -91}.

edited Jun 15, 2010 at 9:06

answered Jun 15, 2010 at 8:52

Matthew Flaschen

286k53 gold badges523 silver badges554 bronze badges

1 Comment

Joachim Sauer Over a year ago

getBytes() uses the platform default encoding iff the no-arguments version is called.

Collectives™ on Stack Overflow

Java InputStream encoding/charset

2 Answers 2

2 Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related