Questions tagged [character-encoding]
Questions about character encoding and decoding. This includes, but is not restricted to, conversion of characters to their numerical value in a character table and vice versa and conversion of characters to strings and vice versa.
123 questions
0
votes
0
answers
49
views
Encoding text from copy and paste
When I copy from a websource some function or expression or slice of code and paste into say wolfram engine, or mathematica, I often get syntax errors. mostly from misinterpretation of the text ...
3
votes
2
answers
197
views
(Wolfram Workbench)"Non-ascii character" warning
Strangely, I still get "Non-ascii character" warning even though I have set my Workspace, Project, and Source code to UTF-8. Moreover,the Non-ascii character is in the commnet as below. the ...
- 1,484
0
votes
0
answers
86
views
Neural Network - catenate layer with an input in the form of <|"x"->{"y","z"}|>
I'm doing a research project that requires me to make a custom neural network, and one major challenge I'm facing is that a few pieces of the input data can have varying dimensionality. Specifically, ...
1
vote
1
answer
122
views
Insert a nonlinear PDE in Mathematica
I am new to Mathematica and trying to learn its language.
I want to solve a non-linear PDE but don't know how to insert it.
Would you please help me or give me some tutorial about it?
- 35
6
votes
1
answer
164
views
Converting Unicode Number Subscripts to Standard Mathematica Subscript Notation
I am importing data
data={"t*α₁","α₁*α₄"}
But unfortunately, all of the subscripts are in Unicode:
...
- 277
2
votes
0
answers
171
views
How can I rasterize expressions with Runic characters in Wolfram Cloud?
Here is my function to create polynomials with a specific format:
...
- 687
8
votes
1
answer
333
views
Unicode symbols not encoded properly in BarcodeImage[] in Mathematica 13.3
Bug introduced in 13.2 or 13.3 and fixed in 14.1
Trying to generate a simple QR code with ← or U+2190. I got varying results: the same in Mathematica versions 12.3 and 13.1 but very different in ...
- 287
4
votes
6
answers
526
views
Convert integer to words of specific length
Assume we give the value from 1 to 26 to alphabets a,b,...,z.
Now given an integer ($n$) i like to generate words of specific length ($l$) that sum of them equals $n$. For instance, if
$n=10$, $l=3$;
<...
- 848
1
vote
2
answers
154
views
Can you recover the original vectors that, when summed together, achieved a particular criteria
I am wondering if it is possible to take the binary strings of length $3$ (tuples) whose sums have weight $2$ (i.e they have $2$ non-zero entries)(binary case, meaning they have two $1$'s entries) $$u ...
- 997
7
votes
1
answer
522
views
How can I export a text file in UTF-8 with a BOM (also known as UTF-8 with signature)?
The default setting "UTF8"/"UTF-8" results in a text file without a byte order mark (BOM)/signature. What if ...
- 71.6k
1
vote
0
answers
108
views
Byte array to String with no internal processing, e.g. 𓃰
UTF8 defines a way to map from bytes to glyphs. In particular, glyphs with code <128 can be 1 byte, glyphs with code <2048 can be 2 bytes, etc. up to 4 bytes. For instance, the white queen <...
- 4,293
0
votes
1
answer
143
views
Having issues creating rules from a CSV import
I'm importing a CSV file with two columns. I've done this with other files, but this one is causing issues when trying to create rules to associate IDs with the titles. Here's a sample of the ...
- 189
9
votes
2
answers
874
views
Does Mathematica have an equivalent to the Python 'translate' function?
In Python you can make a character translation table with 'maketrans', passing it two strings of the same length. Then you can call 'translate' passing a new string and the table you just created. The ...
- 189
2
votes
2
answers
198
views
How can I convert the output of IntegerString[n, "Base64"] back into base 10?
BaseForm[] does not allow conversion to bases above base 36. However, IntegerString[] can convert a number ...
- 2,695
2
votes
1
answer
1k
views
How to copy infinity symbol as ∞ and not as \[Infinity]
FromCharacterCode[8734]
(* ∞ *)
No matter how I copy ∞ whether as plaintext or input text or whether it is inside string like <...
- 32.8k
6
votes
1
answer
287
views
Garbled code when using URLSubmit[] to request CJK web page
When importing content containing Chinese (CJK) characters, correct results can be obtained by using URLread
...
- 185
3
votes
2
answers
157
views
Character encoding of the output from URLExecute
Bug persisting through 13.1.0 [CASE:4932406]
I'm using URLExecute to fetch some data from the arXiv preprint server
but it seems that the encoding of the text I ...
- 3,592
7
votes
1
answer
362
views
Equivalent for \hookrightarrow, ↪
Of course I can simply type the unicode ↪, but is there a operator with no built in meaning that renders like this?
The documentation guide/ArrowsAndArrowLikeForms presumably lists all arrows. If it ...
- 4,293
4
votes
1
answer
239
views
Importing text generates whitespaces between every character
I have a TSV file with string data in it that I want to import. While the TSV format is correctly parsed into a Mathematica table format using ...
- 357
7
votes
1
answer
237
views
Unexpected behavior: missing characters in the code page 936
Bug persisting through 13.1.0
In the CP936.m file which will be loaded by
Internal`LoadEncodingFile@"CP936";
there ...
- 3,780
1
vote
0
answers
94
views
Run a command which has a file parameter when the file path passed contains non-ASCII characters
I want to do run a Linux command from a Mathematica notebook. I see, that Mathematica and my Linux operating system use different character coding.
The problem is that the directories and the file ...
- 2,903
4
votes
0
answers
303
views
Is the fancy script 'C' $\mathscr{C}$ available in Mathematica?
In mathjax \mathscr{C} gives me the character $\mathscr{C}$ that I want. There is a UTF listing for this character. https://www.compart.com/en/unicode/U+1D49E
If I copy form this webpage and paste ...
- 1,697
12
votes
3
answers
1k
views
Bug with Json payload with diacritics for HTTPRequest
Working in the creations of APIs with Mathematica 12.2, I got this problem with diacritics when sending JSON data.
Here is a toy code using httpbin.
...
- 26.5k
3
votes
0
answers
158
views
Why is `$SystemCharacterEncoding` different from the actual OS character encoding on Windows?
I was testing on Windows the command $ wolframscript -code 'Alphabet["Greek"]' -charset UTF8 from the document of ...
- 926
0
votes
1
answer
104
views
Purpose of ToCharacterCode [closed]
Is ToCharacterCode function the same as assigning ASCII decimal to a character?
- 305
9
votes
2
answers
1k
views
Hexadecimal bytes as variable converted to binary bits
I am working with hexadecimal bytes but need to shift them into lists of binary bits for a few operations and then back again.
For example, I would like C12B to turn into {1,1,0,0,0,0,0,1,0,0,1,0,1,0,...
- 193
1
vote
0
answers
99
views
Problem with CharacterEncoding trying to obtain Twitter data
I'm using Mathematica 11 installed in Ubuntu.
I'm trying to retrieve Twitter data these days. So, I'm using:
twitter = ServiceConnect["Twitter"]
Everything is ...
- 11
2
votes
0
answers
168
views
Characters don't print right in WolframScript
Mathematica version: 12.1 & 11.3
Platform: Windows 10
Since windows's cmd is GBK, so my file is GBK.
...
- 10.6k
3
votes
2
answers
304
views
Bug in WebExecute TypeElement with diacritics characters [Solved]
[Solved in 12.1.1]
When using WebExecute TypeElement, I get $Failed if I have a diacritics ...
- 26.5k
1
vote
0
answers
77
views
What are the special characters `\!`, `\@`, etc.? [duplicate]
Character codes 63425 through 63436 (within the WL private use range) can be typed with backslash escape sequences. Despite displaying like digraphs they are single characters:
...
- 684
1
vote
0
answers
84
views
How can I handle box drawing characters?
I must generate files for windows batch scripting, modifying template files, es.
...
4
votes
1
answer
228
views
One-hot vectors encoding with NetEncoder
I have a text and vocabulary (text has two "the" entries):
...
- 5,017
1
vote
0
answers
61
views
Is there an inverse type function to SequenceLastLayer[]?
I am working on a text based autoencoder, and was happy to see I can feed a variable length text sequence into my network and use SequenceLastLayer[] after a GatedRecurrentLayer[] to only get the ...
- 1,467
4
votes
1
answer
155
views
AlphabeticSort of Chinese character is not working now
I want to sort strings of chinese characters. I used to use below code
list={"大家","早上","好"};
AlphabeticSort[list, Entity["Language", "ChineseMandarin"]]
And it ...
- 17.7k
1
vote
0
answers
121
views
How to ImportString "JSON" with Unicode characters? [duplicate]
Update
The short answer is that when getting results back from an API into an HTTPResponse object the "Body" property ...
- 44k
3
votes
3
answers
7k
views
How do I add EOF character to a CSV file?
I am using Mathematica to create a CSV file to be used by another program (SRIM if needed). The file must end with the EOF character. I get an error from SRIM telling me that the file does not have ...
- 6,991
3
votes
3
answers
205
views
Convert Latin to e.g Gothic
Sorry for the very simple question but I can't seem to find an answer anywhere.
I have a list of Latin letters, e.g. {a, b, c, d...}. I want to convert them to the corresponding Gothic letters. How ...
- 185
5
votes
2
answers
362
views
Convert Unicode to Math Input
Any Ideas how I could convert unicode to Mathematica Input ?
For instance : I have the following Unicode: test= {"a", "\\:1d62"}
So here I would like to convert ...
- 3,103
2
votes
0
answers
195
views
How to render an emoji? [duplicate]
Is there any way to get Mathematica to render emoji's from unicode strings? For instance, ":1F600" should be 😀but it renders as an "oulinded question mark":
Also, ...
- 3,880
6
votes
1
answer
195
views
Exporting/Saving Notebook with Non English characters to PDF
On a related question, I now have another problem with saving notebook to PDF.
OS: Windows 7 SP1 English Ultimate
MMA: 11.3 X64
The Chinese characters displays ...
- 5,056
7
votes
1
answer
280
views
Fail to load a UTF8 package with one special Chinese character
Please check out the following code.
BeginPackage["WTF`"]
f::usage="blah"
Begin["`Private`"]
f[x_]:=1(*盘*);
End[]
EndPackage[]
I've save the code to ...
- 2,814
2
votes
0
answers
78
views
How to make the notebook show properly some special characters?
Those special characters, e.g., are ⫺, ⬇, ⫽ and ⬃. They ...
- 10.1k
6
votes
0
answers
477
views
Passing UTF-8 to wolframscript on the command line
I am trying to use Unicode input to wolframscript -c, like so:
$ ./wolframscript -c '1/∞'
â^(-1)
The source of this behavior ...
- 236
19
votes
1
answer
651
views
HMAC correct on 11.2 but wrong and different on 11.3 running same code
If you run this code on 11.2
...
- 1,309
6
votes
1
answer
203
views
Encoding strings to workaround CloudDeployment bugs
Bug introduced with FormFunction and persisting through Cloud V1.48
CASE:4026150 confirmed
There are few problems when deploying simple ...
- 139k
4
votes
2
answers
482
views
Import text outlines from PDF format properly
Bug introduced in 10.0 and persisting through 12.0
I am attempting to implement this: How can I wrap text around a circle?
But I need Unicode support.
Unfortunately, in my case, this gives a broken ...
- 807
0
votes
0
answers
180
views
Chinese characters encoding into numbers and back again to characters problem
I have a bunch of Chinese characters that I would like to change into decimal numbers, then into binary numbers. I would then like to take back the binary numbers and change them into decimal numbers ...
- 199
3
votes
2
answers
167
views
Explicitly control newline behaviour when writing strings with arbitary encoding
On Windows system, Mathematica by default opens stream which will automatically transform any "\n" into "\r\n". But I want pure ...
- 17.7k
1
vote
0
answers
113
views
2
votes
1
answer
1k
views
How to type these Unicode characters or make my own character?
How can I type the following characters without copying and pasting them from the web? Perhaps define them as a character myself?
Also, they are not displayed properly when pasted in a code cell but ...
- 1,923