Node Buffer Alias - binary is latin1?

Question

According to this page:

'binary' - Alias for 'latin1'.

However, binary is not representable in latin1 as there are certain codepoints which are missing. So, a developer like me who wants to use NodeJS buffers for binary data (an extremely common use-case) would expect to use 'binary' as the encoding. There doesn't appear to be any documentation that properly explains how to handle binary data! I'm trying to make sense of this.

So my question is: Why was latin1 chosen as an alias for binary?

People have mentioned that using null as the encoding will work for binary data. So a followup question: Why doesn't null and 'binary' do the same thing?

ottomeister · Accepted Answer · 2018-11-29 04:21:11Z

4

The Node documentation's definition of 'latin1', on the line above the definition of 'binary' cited in the question, is not ISO 8859-1. It is:

'latin1' - A way of encoding the Buffer into a one-byte encoded string (as defined by the IANA in RFC1345, page 63, to be the Latin-1 supplement block and C0/C1 control codes).

The 'latin1' charset specified in RFC 1345 defines mappings for all 256 codepoints. It does not have the holes that exist at 0x00-0x1f and 0x7f-0x9f in the ISO 8859-1 mapping.

Why doesn't null and 'binary' do the same thing?

Node does not have a null encoding. If you call Buffer.from('foo', null) then you get the same result as if you had called Buffer.from('foo'). That is, the default encoding is applied. The default encoding is 'utf8', and clearly that can produce results that differ from the 'binary' encoding.

answered Nov 29, 2018 at 4:21

ottomeister

5,8582 gold badges26 silver badges27 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

James Watkins Over a year ago

Then can you explain why fs.createReadStream has a default encoding of null? nodejs.org/api/fs.html#fs_fs_createreadstream_path_options

Paul Draper Over a year ago

@JamesWatkins in that context the encoding is optional. I.e. no encoding gives you Buffers and an encoding gives you strings. Buffer.from(string) needs an encoding. If you don't provide one, it uses utf8.

Collectives™ on Stack Overflow

Node Buffer Alias - binary is latin1?

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related