encode string as utf-16 to base64 in javascript

Question

I'm struggling to find any resources on this online, which is concerning. I've been reading about UCS-2 and UTF-16 woes, but I can't find a solution.

I need to get a value from an input:

var val = $('input').val()

and encode it to base64, treating the text as utf-16, so:

this is a test

becomes:

dABoAGkAcwAgAGkAcwAgAGEAIAB0AGUAcwB0AA==

and not the below, which you get treating it as UTF-8:

dGhpcyBpcyBhIHRlc3Q=

What's the question? How to generate the same as in UTF-8? I suppose not — Alexander
– Alexander, Commented May 19, 2013 at 9:24

Community · Accepted Answer · 2017-03-20 10:29:34Z

1

Your data, once read into JavaScript, will be in an encodingless numerical format (strictly speaking, it has to be in Unicode Normalised Form C, but Unicode is just a series of identifying numbers for each glyph in the Unicode lexicon. It's encoding-less). So: if you specifically need the data encoded as a UTF-16 byte sequence, do so, then base64 encode that.

But here's the fun part: which UTF-16 do you need? Little or Big Endian? With or without BOM? UTF-16 is a really inconvenient encoding format (we're not even going to touch UCS-2. It's obsolete. Has been for a long time).

What you really should need is to get a text value from your HTML element, Base64 encode its value, and then have whatever receives that data unpack it as UTF8; don't try to make JavaScript do more work than it has to. I presume you're sending this data to a server or something, in which case: your server language is way more elaborate than JavaScript, and can unpack text in about a million different encodings thanks to built-in functions. So just use that. Don't solve Y for X.

edited Mar 20, 2017 at 10:29

CommunityBot

11 silver badge

answered May 19, 2013 at 14:45

Mike 'Pomax' Kamermans

54.3k17 gold badges134 silver badges182 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Andrew Bullock Over a year ago

I need to accept high bit characters (like Chinese characters). UTF-8 wont handle this, will it?

Mike 'Pomax' Kamermans Over a year ago

UTF8 is an encoding for Unicode, not "part of it", so yeah it will handle it just fine. That's what it was designed for =) Fun fact: UTF16 and UTF32 use 16 and 32 bits per character. UTF8 does not use 8 bits per character, instead it uses 'as many 8 bit blocks as necessary'

Triynko Over a year ago

"What you really should need is to get a text value from your HTML element, Base64 encode its value". You can't base64 encode text. You can only base64 encode a byte array, so there is some implicit encoding of the text to a byte array that must be made explicit. You much choose a byte-encoding for the unicode characters first (i.e. UTF8, UTF16, etc.) and once you encode all the characters into a byte array, then base64 encode that.

Mike 'Pomax' Kamermans Over a year ago

As a correction on that, you can base64-encode any byte sequence, including text. This is trivially checked by opening a console and typing btoa("cats"), or even btoa(3). Since these inputs are just bytes as well as what you as human think them to be, they will get base64 encoded just fine. The only thing that truly matters is that the receiver knows what encoding the original data is in, so it do the right thing later (and then only if one needs a not-UTF8 data source, really)

Collectives™ on Stack Overflow

encode string as utf-16 to base64 in javascript

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related