Questions tagged [text-encoding]
The text-encoding tag has no summary.
26 questions
5
votes
9
answers
3k
views
Methods to increase the amount of data sent in a packet
I have been working on launching high-altitude balloons (HABs, or weather balloons) and I have been using LoRa to enable long-range communication with my balloons. It's been great and pretty reliable, ...
- 75
-2
votes
1
answer
4k
views
How to identify encoding of a text string? [closed]
I guess most of you already met them. You get them from your data sources, see them in your logs, or in the output from your legacy systems. Some strings you can't really read.
To derive any useful ...
- 123
2
votes
2
answers
571
views
Compressing EBCDIC file vs UTF8
Today I went across a weird case for which I have no explanation, so here I am.
I have two files with identical content, but one is encoded in UTF-8 and the other one is in IBM EBCDIC. Both of them ...
- 137
5
votes
1
answer
428
views
UTF-8 questions
When you encode a code point to code units based on UTF-8, then if the code point fits on 7 bits, the most significant bit is set to zero so that it tells you it is a character which is stored on 1 ...
3
votes
4
answers
3k
views
Reduce number of digits by converting to alphanumeric data
We have an app that receives a web service request, processes it and sends it back to our client by another web service call. There is a unique field in the request, a tracking Id, which currently ...
0
votes
1
answer
91
views
Name of data structure where a string determines its value
A few times in my career I've found myself writing decoders for responses from IoT products or weird apis that insted of using JSON or XML as a response, they reply with something like ...
1
vote
1
answer
91
views
Dictionary Coding for OLAP
Let's say we have a generic table like below:
id, name, price, quantity
20 product_x 5,00 100
20 product_y 5,00 100
20 ...
user313675
4
votes
4
answers
4k
views
Windows compatibility with Unix/Linux newline "\n"
A follow-up to Difference between '\n' and '\r\n'.
It's been few decades since the schism was introduced. Nowadays, when documents are being exchanged over the internet, typically ...
- 349
10
votes
1
answer
3k
views
What is the encoding used in Git's binary patches?
Git can generate patches/diffs for binary files as well as for text files.
I'm trying to figure out what encoding it uses for its binary patches.
Here is an example:
diff --git a/www/images/...
- 447
-1
votes
2
answers
317
views
How to find charset name by byte codes?
I have a printer and SDK to work with it in Java. Printer working well with english letters and digits but doesn't print correctly special symbols like 'ä' or 'ê'.
I suppose that I need to convert ...
- 107
-1
votes
2
answers
9k
views
Fast and simple hex compression
I'm working on a project that requires a TCP connection between a client and server. The current protocol encodes the data into hex and then sends it. However, hex increases the length of the payload ...
- 155
2
votes
1
answer
1k
views
What type of encoding does wikipedia use?
I have the wikipedia data dump and trying to decode special characters in the page titles, except a lot of characters don't match up the "standard" ascii encoding (referencing from here.)
As an ...
- 31
4
votes
2
answers
417
views
Why does ISO 8859-1 contain letter-free diacritics?
ISO 8859-1 contains a few letter-free diacritics: The diaeresis (¨), the acute accent (´), the cedilla (¸) and the macron (¯).¹
Why were they included? As far as I know (please correct me if I am ...
- 9,868
7
votes
2
answers
10k
views
How is encoding handled correctly during copy-paste between programs?
Suppose
a program A opens a text file A using encoding A to decode the file, and
a program B opens a text file B using encoding B.
When we copy some text from file B in program B to file A in ...
- 5,565
1
vote
2
answers
77
views
cross-application text formatting
I am not sure whether this question is a good fit for this site, but if it is not, please let me know and I will take it down.
If it is off-topic, some general info on where I can look for these ...
- 273
21
votes
4
answers
4k
views
Why does UTF-8 waste several bits in its encoding
According to the Wikipedia article, UTF-8 has this format:
First code Last code Bytes Byte 1 Byte 2 Byte 3 Byte 4
point point Used
U+0000 U+007F 1 0xxxxxxx
U+0080 U+...
- 321
0
votes
1
answer
5k
views
VB.NET - Serial Key Generating (Encryption, Hashing, Encoding?) Algorithm
I am trying to create a basic licensing system where I take a unique ID from the client computer, and I get this Hexadecimal string (hyphens removed e.g. "84-18-CE-...."):
"...
- 103
2
votes
2
answers
6k
views
When is it beneficial to not use utf-8? [duplicate]
When is it beneficial to use encodings other than UTF-8? Aside from dealing with pre-unicode documents, that is. And more importantly, why isn't UTF-8 the default in most languages? That is, why do I ...
- 1,483
4
votes
4
answers
19k
views
What encoding is used by javax.xml.transform.Transformer? [closed]
Please can you answer a couple of questions based on the code below (excludes the try/catch blocks), which transforms input XML and XSL files into an output XSL-FO file:
File xslFile = new File("...
- 141
8
votes
2
answers
16k
views
Does FFMpeg support gpu acceleration of media encoding/decoding?
I was wondering if ffmpeg supported gpu acceleration. I was reading on their websites and came across contradicting information.
http://www.ffmpeg.org/general.html#Video-Codecs
-H.264 / AVC / MPEG-4 ...
- 143
3
votes
2
answers
1k
views
Unable to debug an encodded javascript?
I’m having some problems debugging an encoded javacscript. This script I’m referring to given in this link over here.
The encoding here is simple and it works by shifting the unicodes values to ...
- 31
8
votes
2
answers
2k
views
How relevant is UTF-7 when it comes to parsing emails?
I recently implemented incoming emails for an application and boy, did I open the gates of hell? Since then every other day an email arrives that makes the app fail in a different way.
One of those ...
- 313
2
votes
2
answers
895
views
What is really happening when we change encoding in a string?
http://php.net/manual/en/function.mb-convert-encoding.php
Say I do:
$encoded = mb_convert_encoding ($original);
That looks like simple enough. WHat I am imagining is the following
$original has a ...
- 739
4
votes
3
answers
3k
views
URL Encryption vs. Encoding
At the moment non/semi sensitive information is sent from one page to another via GET on our web application. Such as user ID or page number requested etc. Sometimes slightly more sensitive ...
- 313
21
votes
4
answers
61k
views
What type of encoding can I use to make a string shorter?
I am interested in encoding a string I have and I am curious if there is a type of encoding that can be used that will only include alpha and numeric characters and would preferably shorten the number ...
- 645
41
votes
8
answers
5k
views
Why are there multiple Unicode encodings?
I thought Unicode was designed to get around the whole issue of having lots of different encoding due to a small address space (8 bits) in most of the prior attempts (ASCII, etc.).
Why then are there ...
- 1,657