Why java.lang.StringEncoding.encode ignore given charset to use default one?

Question

My app default encoding is set to "UTF-8" (using -Dfile.encoding=UTF-8 on startup). When I use the String class method "getBytes(String charsetName)" with charset="ISO-8859-1", it appears that StringCoding.encode finally uses the default charset (UTF-8) instead of the given one (ISO-8859-1).

For an unknown reason, I can debug step by step on this method, but I'm not able to inspect inner elements value (only parameters which are named arg0, arg1 ...)

In java 1.6.10, StringCoding.encode is written :

static byte[] encode(String charsetName, char[] ca, int off, int len)
    throws UnsupportedEncodingException
{
    StringEncoder se = (StringEncoder)deref(encoder);
    String csn = (charsetName == null) ? "ISO-8859-1" : charsetName;
    if ((se == null) || !(csn.equals(se.requestedCharsetName())
                  || csn.equals(se.charsetName()))) {
        se = null;
        try {
        Charset cs = lookupCharset(csn);
        if (cs != null)
            se = new StringEncoder(cs, csn);
        } catch (IllegalCharsetNameException x) {}
        if (se == null)
            throw new UnsupportedEncodingException (csn);
        set(encoder, se);
    }
    return se.encode(ca, off, len);
}

With step by step debug, I never enter the if block and then no new StringEncoder with my ISO-8859-1 charset is created. Finally, the Charset.defaultCharset() method is called.

Any clues ? Thanks

Is the -Dfile.ecoding=UTF-8 actually spelt like that? If so it should be -Dfile.encoding=UTF-8 — stark
– stark, Commented Jun 20, 2011 at 10:28
Could you post a complete example (SSCCE) where this occurs? — Paŭlo Ebermann
– Paŭlo Ebermann, Commented Jun 20, 2011 at 10:31

Jon Skeet · Accepted Answer · 2011-06-20 10:32:13Z

3

If you're not getting into the if block, then this expression must be false

(se == null) || !(csn.equals(se.requestedCharsetName())
                  || csn.equals(se.charsetName()))

That means that:

se mustn't be null
The second part must be true before hitting the !, so one of these subexpressions must be true:
- csn.equals(se.requestedCharsetName())
- csn.equals(se.charsetName())

In other words, se is already appropriate for the charset name you've requested.

This isn't using the "default" encoding for the VM; it's using the encoder last used in this thread.

I highly doubt you've found a JRE bug - it looks okay to me. So what caused you to debug into this to start with? Can you provide a short but complete program which demonstrates an error using this? Something encoding to the wrong bytes?

answered Jun 20, 2011 at 10:32

Jon Skeet

1.5m893 gold badges9.3k silver badges9.3k bronze badges

Sign up to request clarification or add additional context in comments.

13 Comments

Redmat Over a year ago

Of course it's not a jre bug, there must been something I haven't understand.

Jon Skeet Over a year ago

@Redmat: So do you have a short but complete program which demonstrates String.getBytes("ISO-8859-1") giving the wrong result?

Redmat Over a year ago

I'm working on it :) Actually, with a very short program, I'm not able to reproduce the bug. I think that the line StringEncoder se = (StringEncoder)deref(encoder); is the key of my issue.

Redmat Over a year ago

The reason why I get into this is because with my file-encoding set to UTF-8 for my application and my target charset - for the functionnality i have to code - det to ISO-8859-1, I obtain unreadable bytes when writing my result into a File. If I open this file with PSPad hex, a lot of byte are replaced by 0x3F, which correspound to '?' in ISO-8859-1

Jon Skeet Over a year ago

@Redmat: That suggests you've got characters in your string which aren't representable in ISO-8859-1, which is entirely feasible...

|

stark · Accepted Answer · 2011-06-20 10:29:11Z

0

You need to change the

-Dfile.ecoding=UTF-8 to

-Dfile.encoding=UTF-8

answered Jun 20, 2011 at 10:29

stark

8395 silver badges8 bronze badges

Comments

A.Grandt · Accepted Answer · 2013-12-16 13:17:57Z

0

encode should never ignore the given charset to revert to the one given in -Dfile.encoding.

It does. As does decode by the way, even if the source does show that it finds the charset, and sets it in the line:

set(encoder, se);

Neither encode or decode are threadsafe, so the value is bound to be overridden by the system default, it may be before or after decode manages to use the set value though.

Imho, this is a bug in the JRE. True the OP had a typo, but that does not change the fact that if you ask String to decode a byte array to UTF-8, it should always return UTF-8, and not silently fall back to something else.

answered Dec 16, 2013 at 13:17

A.Grandt

2,2622 gold badges20 silver badges23 bronze badges

Collectives™ on Stack Overflow

Why java.lang.StringEncoding.encode ignore given charset to use default one?

3 Answers 3

13 Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

13 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related