Wrong encoding for tomcat/spring server REST input

Question

I have a REST service made with Tomcat 8 and Spring v5.3.18. The service gets and stores a string in DB.

I have an issue with this input:

{"name": "Pawe\u0142"}

It is a valid UTF8 char. The string saved is Pawe? so there is something wrong...

I started to check where it can be the missing encoding, but I found that it depends on the tomcat environment. Two servers with the same software, linked to a common database but with different operative system, first is Linux, second is Windows have different responses.

The Windows server is working and store the valid UTF8 char, the linux server is not working and stores the char as "?".

So I deduced it is not related to the software or missing encoding in sw configuration, but probably it is related to tomcat or java.

Do you have any idea about what can be wrong?

the linux server is not working and stores the char as "?". In the file in which it's stored (let's call it 'file.txt') can you please do this at the command line and show what's printed. grep 'Pawe' file.txt | xxd — g00se
– g00se, Commented Aug 24, 2022 at 16:31

LMC · Accepted Answer · 2022-08-24 19:12:08Z

2

Probable causes could be LC_CTYPE linux locale or file.encoding JVM property:

Linux LC_CTYPE (or LC_ALL) locale is set to other than UTF-8. Simulated as

LC_CTYPE="ISO-8859-1" jshell

jshell> import java.nio.charset.Charset;

jshell> Charset.defaultCharset()
$2 ==> US-ASCII

jshell> String a = "adf \u0142";
a ==> "adf ?"

JVM has file.encoding property set to other than UTF-8

jshell -J-Dfile.encoding="ISO-8859-1" --execution="local"

jshell> import java.nio.charset.Charset;

jshell> Charset.defaultCharset()
$2 ==> ISO-8859-1

jshell> String a = "adf \u0142";
    a ==> "adf ?"


jshell -J-Dfile.encoding="UTF-8" --execution="local"

jshell> String a = "adf \u0142";
a ==> "adf ł"

Linux quick tests

java -Dfile.encoding="ISO-8859-1" -XshowSettings:properties -version 2>&1 | grep 'encoding'
    file.encoding = ISO-8859-1
    sun.io.unicode.encoding = UnicodeLittle
    sun.jnu.encoding = UTF-8

java -XshowSettings:properties -version 2>&1 | grep 'encoding'
    file.encoding = UTF-8
    sun.io.unicode.encoding = UnicodeLittle
    sun.jnu.encoding = UTF-8

ps -lf -C jshell | grep 'file.encoding'
0 S lmc      7266  2225 10  80   0 - 895886 -     14:48 pts/2    00:00:05 jshell -J-Dfile.encoding=ISO-8859-1 --execution=local


LC_ALL="es_ES.ISO-8859-1" java -help 2>&1 | tail -n-3
Para especificar un argumento para una opci�n larga, puede usar --<nombre>=<valor> o
--<nombre> <valor>.

edited Aug 24, 2022 at 19:12

answered Aug 24, 2022 at 16:55

LMC

14.4k3 gold badges34 silver badges62 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Tobia Over a year ago

Thanks for the tip, finally the environment setting was right, the error was in DB connection

Tobia · Accepted Answer · 2022-08-25 10:04:32Z

0

Finally the error was in the connection string with DB / JDBC.

I had to append some params to the connection string URL for Mysql driver:

dbConnectionString=jdbc:mysql://localhost/test?useSSL=false&useUnicode=true&characterEncoding=utf-8

answered Aug 25, 2022 at 10:04

Tobia

9,60929 gold badges122 silver badges245 bronze badges

Collectives™ on Stack Overflow

Wrong encoding for tomcat/spring server REST input

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related