javascript compression algorithm that supports BINARY data?

Question

I'm looking for a lossless compression algorithm (like LZW or Huffman or anything) in javascript, that accepts and returns raw, binary data.

With 'binary data' I mean a sequence of bytes, which may come in any of the following forms:

a string containing characters with any value from 0 to 255
an array containing integers with any value from 0 to 255
a string containing a hexadecimal representation of the data (i.e. 2 hex digits per byte)
a string containing the base64 encoded representation of the data
or anything else that can be unambiguously converted from or to any of the above

Now obviously there are TONS of javascript implementations available everywhere, for a wide range of algorithms. However EVERYTHING I find seems to do crazy stuff like:

returning an array containing also values >255 (so what is the compression ratio now? how do I represent this in bytes, or how would I go about saving this to a file for example?)
messing with character encodings in strings, converting from/to unicode or url/html entities or whatnot (it's BINARY, character encoding does not apply here!)
return other representations that don't seem suitable for binary storage (i.e. cannot be converted to sequence of bytes)

Would anyone know of a good javascript compression (+decompression) implementation that suits my binary fetish?

Keep in mind that it's only been recently that "low level" programmers started noticing JavaScript, and even more so that fast JITs capable of processing nontrivial data even existed. Likewise, be sure to test well, because performance varies wildly even between modern browsers. — Potatoswatter
– Potatoswatter, Commented Mar 29, 2012 at 11:06
Thanks, will do. I typically expect to be dealing with chunks of data of up to 10 or 15 KB, i.e. not that much, so I'd expect clients to be able to compress this without much delay. Reason I'm using compression is that I expect LOTS of clients submitting such chunks of data simultaneously, so anything that can lower the server load (by compressing the individual chunks client side) will help me. — Sheldon Pinkman
– Sheldon Pinkman, Commented Mar 29, 2012 at 11:19
That is a lot of data for JavaScript. This is a language where arrays are just dictionaries with numeric keys. So a 10,000 element array could be a 10,000 entry hashtable with a naive JS implementation and a crappy JIT. I looked at a similar problem just a year ago and sending binary data at all seemed to be cutting edge then. I see more online resources now, but you might be ahead of the curve. — Potatoswatter
– Potatoswatter, Commented Mar 29, 2012 at 12:21
Well more than 10 KB will be rare, and I've already tested with several algorithms, and they seemed to perform the compression instantly (except they output the compressed result in a way that ain't easy to convert to a minimal byte sequence or base64 string) — Sheldon Pinkman
– Sheldon Pinkman, Commented Mar 29, 2012 at 14:37
Well, Firefox runs Highcharts over tens of thousands of datapoints a lot faster than Safari, for what it's worth. Looking around now I saw a lot about the ArrayList class from WebGL library. It's apparently the new great thing for binary JavaScript (and includes an AJAX interface) so you might look for a library using that. — Potatoswatter
– Potatoswatter, Commented Mar 29, 2012 at 16:33

Sheldon Pinkman · Accepted Answer · 2012-03-30 00:01:25Z

1

I think I found what I was looking for after all: this deflate + inflate implementation in javascript seems to work with strings as byte sequences.

answered Mar 30, 2012 at 0:01

Sheldon Pinkman

1,1164 gold badges15 silver badges21 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Dasarp · Accepted Answer · 2012-03-29 10:33:51Z

-3

first of all create a closure for hold the binar or hex or decimal flags

function ASearch() { }
ASearch.Flag = {
    Front_Wheel_Drive: 0xF, Rear_Wheel_Drive: 0xF0, Four_Wheel_Drive: 0xF00,
    Auto: 0xFF, Manual: 0xFF00,
    Gas: 0xF, Diesel: 0xF0, Hybrid: 0xF00, Electrical: 0xF000,
    Two: 1, Three: 2, Four: 4, Five: 8, Six: 16, Eight: 32, Ten: 64, Twelve: 128
};

then set like this

SetFlag = (function (e) {
   e = e.srcElement;
   $("#" + e.parentNode.name).val(e.checked ?
        $("#" + e.parentNode.name).val() | ASearch.Flag[e.id] :
        $("#" + e.parentNode.name).val() ^ ASearch.Flag[e.id]);
});

this is an example for packed data in a 32 bit integer

there are four variable... i've used them for 18 flags.. this is fast and super effective

for example...

int i = 0; //binary = 0000 0000 0000 0000

i = i | 255; //binary = 0000 0000 1111 1111

i = i ^ 255; //binary = 0000 0000 0000 0000

i = i | 0xFF00; //binary = 1111 1111 0000 0000

i = i | 255; //binary = 1111 1111 1111 1111

i = i ^ 0xFF00; //binary = 0000 0000 1111 1111

edited Mar 29, 2012 at 10:33

answered Mar 29, 2012 at 10:27

Dasarp

1941 silver badge5 bronze badges

2 Comments

Sheldon Pinkman Over a year ago

Ehhr.. I fail to see how this is related to my question? I understand how to work with bits or binary flags. It's just that compression algorithm return, for example, something like [45,281,12,1134,98] which does not automatically translate to a sequence of bytes (well, I could store each integer a 4 bytes of course, but that kinda defeats the whole purpose of the compression).

Sheldon Pinkman Over a year ago

Sorry, but I really don't see where you're going. I understand about bits, and binary flags and operators. Really, I do. My question is about compression algorithms, and their way of returning data that seems unsuitable for storing as a byte sequence efficiently. There's nothing with flags or srcElement or parents or nodes involved whatsoever.

Collectives™ on Stack Overflow

javascript compression algorithm that supports BINARY data?

2 Answers 2

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related