base64 encode binary string returns different values in NodeJS and Python

Question

I am trying to base64 encode a binary string in NodeJS and python and I'm getting 2 different values.

Note that the value is i is 16 random bytes generated in python using os.urandom(16)

NodeJS

> var i = '>e\x93\x10\xabK\xbe\xfeX\x97\x9a$\r\xef\x8f3';
> var s = new Buffer(i).toString('base64');
> console.log(s);
PmXCkxDCq0vCvsO+WMKXwpokDcOvwo8z

Python

>>> import base64
>>> i = '>e\x93\x10\xabK\xbe\xfeX\x97\x9a$\r\xef\x8f3'
>>> s = base64.b64encode(i)
>>> print s
PmWTEKtLvv5Yl5okDe+PMw==

Am I doing something wrong? It does work for regular string such as my name.

NodeJS

> var s = new Buffer('travis').toString('base64');
undefined
> console.log(s);
dHJhdmlz

Python

>>> s = base64.b64encode('travis')
>>> print s
dHJhdmlz

Martijn Pieters · Accepted Answer · 2013-10-01 20:37:42Z

10

NodeJS is encoding the UTF-8 representation of the string. Python is encoding the byte string.

In Python, you'd have to do:

>>> i = u'>e\x93\x10\xabK\xbe\xfeX\x97\x9a$\r\xef\x8f3'
>>> i.encode('utf8').encode('base64')
'PmXCkxDCq0vCvsO+WMKXwpokDcOvwo8z\n'

to get the same output.

You created the buffer using a default encoding, which means it interpreted i as UTF-8 to begin with. You need to tell Buffer to treat i as binary instead:

> var i = '>e\x93\x10\xabK\xbe\xfeX\x97\x9a$\r\xef\x8f3';
> var s = new Buffer(i, 'binary').toString('base64');
> s
'PmWTEKtLvv5Yl5okDe+PMw=='

edited Oct 1, 2013 at 20:37

answered Oct 1, 2013 at 20:29

Martijn Pieters

1.1m326 gold badges4.2k silver badges3.4k bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

tbeauvais Over a year ago

Thanks! It seems I was experiencing this because I was hardcoding the value of i for testing. But generating it properly works.

user2226755 Over a year ago

Thanks! I spent two days to find the issue of my hash function, I never thought about different result between nodejs and other languages like python/php/perl... We are getting the same with hash crypto.createHash("sha1").update(data, "binary"), I don't understand why we don't have warning in the documentation^^".

Collectives™ on Stack Overflow

base64 encode binary string returns different values in NodeJS and Python

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related