Go and Node.js base64 encoding, decoding problems

Question

I am base64 encoding a string in Go

Then decoding in javascript (I have tried 3 different methods)

I cannot get the javascript result to match the original Go string

Go (encode)

a := []byte {138,143,163,224,178,73,161,15,240,121,53,192,198,182,52,245}
fmt.Println("a", string(a), a, len(a))
b := base64.StdEncoding.EncodeToString(a)
fmt.Println("b", b, []byte(b), len([]byte(b)))

js (decode)

 const b = [105,111,43,106,52,76,74,74,111,81,47,119,101,84,88,65,120,114,89,48,57,81,61,61];
 let bString = aesjs.utils.utf8.fromBytes(b);
 console.log("b", bString, b, b.length);

 let a1String = atob(bString);
 let a2String = Base64.decode(bString);
 let a3String = Buffer.from(bString, 'base64').toString('utf8');
 let a1 = aesjs.utils.utf8.toBytes(a1String);
 let a2 = aesjs.utils.utf8.toBytes(a2String);
 let a3 = aesjs.utils.utf8.toBytes(a3String);
 console.log("a", a1, a1.length, a2, a2.length, a3, a3.length);

All 3 methods fail, i.e. a1 != a, a2 != a, a3 != a

I assumed base64 encoding/decoding would be simple

What am I missing? Thanks

EDIT: The js code had a 'typo' - but the problem remains, that ai != a, even a1 != a2 == a3

EDIT: If the original a is simple, then everything works. But when a contains non-printable chars, I cannot decode in js Example where it works: a := []byte {65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65} Here it does not works: a := []byte {138, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65}

Your go code is correct. So the problem is on the JS side.

colm.anseo
– colm.anseo

2019-04-17 21:56:55 +00:00
Commented Apr 17, 2019 at 21:56 — colm.anseo
– colm.anseo, Commented Apr 17, 2019 at 21:56

colm.anseo · Accepted Answer · 2019-04-17 23:42:58Z

2

From: https://github.com/ricmoo/aes-js/blob/master/README.md

UTF8 should NOT be used to store arbitrary binary data as it is a string encoding format, not a binary encoding format

So don't used utf8 related functions to convert raw (non utf8) binary data.

edited Apr 17, 2019 at 23:42

answered Apr 17, 2019 at 22:19

colm.anseo

22.7k5 gold badges50 silver badges64 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

imran Over a year ago

And what is the solution then?

Mark · Accepted Answer · 2019-04-18 04:26:46Z

1

In Go, strings are UTF8:

in := "Good afternoon, or こんにちは"
buf := []byte(in)
str := base64.StdEncoding.EncodeToString(buf)
fmt.Println(str)

That prints the base64 encoded (ASCII safe) representation: R29vZCBhZnRlcm5vb24sIG9yIOOBk+OCk+OBq+OBoeOBrw==

JS strings are UTF-16. So you need to both decode from base64 and convert from utf8 to utf-16. You could use a library, or a helper function, for example:

function b64DecodeUnicode(str) {
    return decodeURIComponent(atob(str).split('').map(function(c) {
        return '%' + ('00' + c.charCodeAt(0).toString(16)).slice(-2);
    }).join(''));
}

var str = "R29vZCBhZnRlcm5vb24sIG9yIOOBk+OCk+OBq+OBoeOBrw==";
var result = b64DecodeUnicode(str);
"Good afternoon, or こんにちは"

The b64DecodeUnicode() function is copied from this answer, see also the link to MDN for detailed info.

answered Apr 18, 2019 at 4:26

Mark

7,1511 gold badge43 silver badges39 bronze badges

1 Comment

imran Over a year ago

This works for proper UTF8 strings, so I will mark it as correct. Unfortunately, it does not solve my problem, which was not in scope here though: My data that gets encoded is binary (result of an encryption), and I think here the comment of @colminator might apply

Collectives™ on Stack Overflow

Go and Node.js base64 encoding, decoding problems

2 Answers 2

1 Comment

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related