Hex string to ASCII conversion with errors?

Question

I am trying to write a python script to convert a hex string into ASCII and save the result into a file in .der cert format. I can do this in Notepad++ using the conversion plugin, but I would like to find a way to do this conversion in a python script from command line, either by invoking the notepad++ NppConverter plugin or using python modules.

I am part way there, but my conversion is not identical to the ASCII ouptut seen in notepad++, below is a snippet of the output in Notepad++

But my python conversion is displaying a slightly different output below

As you can see my script causes missing characters in the output, and if i'm honest I don't know why certain blocks are outlined in black. But these missing blocks are needed in the same format to the first picture.

Here's my basic code, I am working in Python 3, I am using the backslashreplace error control as this is the only way I can get the problematic hex to appear in the output file

result = bytearray.fromhex('380c2fd6172cd06d1f30').decode('ascii', 'backslashreplace')

text_file = open("C:\Output.der", "w")
text_file.write(result)
text_file.close()

Any guidance would be greatly appreciated.

Those black characters look like the ones which are non-ASCII characters (or are ASCII control-codes). Here's a chart, — martineau
– martineau, Commented Dec 12, 2017 at 15:06
I have adjusted the original question to include the hex string for the given output pictures, thanks. — MikG
– MikG, Commented Dec 12, 2017 at 15:07
@martineau, agreed that was my initial understanding, but I guess it's more around the format, if I can get notepad++ to display a python script output in the same way it does using it's plugin, then I am confident the cert file generated will work. — MikG
– MikG, Commented Dec 12, 2017 at 15:11
When I run your code I get an output file containing the hex bytes 38 0C 2F 5C 78 64 36 17 2C 5C 78 64 30 6D 1F 30. What do want the output to be? I don't care what it looks like in some text editor, like notepad++ or any other—just exactly what you want in the file. The two 5C 78 substrings each represent \x prefixes, by-the-way. — martineau
– martineau, Commented Dec 12, 2017 at 15:34

Pak Uula · Accepted Answer · 2017-12-12 16:02:45Z

2

MikG, I would say that python did exactly what you requested.

You told to convert the bytes to string, and replace bytes with most significant bit set with escape sequence (except for \xFF char).

Characters \x04 (ETB) and \x1F (US) are perfectly legal ASCII chars (though non-printable), and they are encoded using their literal value.

Characters \xd6 and \xd0 are illegal in ASCII - they are 8-bit long. They are encoded using 4-letter long escape sequence, as you asked: "\" (backslash char) and "xd6" / "xd0" strings

I'm not good with DER, but suppose that you expect to have raw 8-bit sequences. Here is how this could be accomplished:

result = bytearray.fromhex('380c2fd6172cd06d1f30')

with open("Output.der", "wb") as text_file:
    text_file.write(result)

Please note "wb" specifier to open -- it tells python to do binary IO.

I also used with statement to ensure that text_file is closed whatever happens with write.

edited Dec 12, 2017 at 16:02

answered Dec 12, 2017 at 15:57

Pak Uula

3,5051 gold badge14 silver badges18 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

MikG Over a year ago

So close! it's almost there with your solution, I am now just missing the NUL blocks (similar to the black blocks in the diagrams), instead I get a blank space in the output when opened in Notepad++. NUL can be added by adding 00 in the original hex string.

Mark Ransom Over a year ago

@MikG I don't have any problems with NUL characters.

Pak Uula Over a year ago

@MikG it is the feature of a text editor that you use to view the DER file. It reads up to the first NUL char. Use a hex editor instead. Or simply check the size of the output file. I tried in Linux python 3.6 - it works fine.

MikG Over a year ago

Thanks @PakUula you are correct, the NUL entries are there. I copy and pasted the output to the same text editor window as the Notepad++ generated conversion to do a visual comparison and the NUL entries got omitted on pasting for some reason, but they are there in the file created from your script, many thanks for your help.

Collectives™ on Stack Overflow

Hex string to ASCII conversion with errors?

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related