Python urldecode urlencoded bytecode data

Question

Payload contains an HMAC tag as well as a nonce for AES. Client-side printing the tag and nonce result in (for example):

#tag:  b'=x\x9d{_0\xf9;c8\x94inc]\xb1'
#nonce:  b'\x1f\xf4\xbe\xcc\xf2\x84f\xf2*\x8dP\x16\xc8\x02\xfe\xbe'

requests.post(url, data=payload, headers={"Content-Type": "application/octet-stream"}, verify="myShnazzyCertificate.pem")

Server-side, my flask api route receives a tag and nonce that have evidently been urlencoded:

data = flask.request.data
## stuff happens here, then -> print("tag: ", tag); print("nonce: ", nonce)
#tag:  b'%3Dx%9D%7B_0%F9%3Bc8%94inc%5D%B1'
#nonce:  b'%1F%F4%BE%CC%F2%84f%F2%2A%8DP%16%C8%02%FE%BE'

How do I remove the urlencoding (or prevent it from happening?) while keeping the tag and nonce as bytecode? I tried:

tag = tag.replace(b"%", bytes(r"\x".encode("utf-8")))
nonce = nonce.replace(b"%", bytes(r"\x".encode("utf-8")))

But HMAC verification failed since the tag has "{" and the nonce has "*" which also got encoded, so I'd need something more exhaustive.

Kache · Accepted Answer · 2024-11-06 00:40:19Z

0

Although there may be a way to use "Content-Type": "application/octet-stream" to send binary parameters, I've always considered it to be intended for files.

I suggest not sending binary params via HTTP (a "text protocol") by converting the binary to url-safe text and back, e.g. base64

answered Nov 6, 2024 at 0:40

Kache

17k13 gold badges59 silver badges87 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Ryan Farber · Accepted Answer · 2024-11-06 01:27:09Z

0

Cheers to @Kache for mentioning what should have been obvious: just base64 encode to send plaintext.

For future reference, this is how I did so.

Client-side:

tag   = base64.b64encode(tag)
nonce = base64.b64encode(nonce)
# tag and nonce are packed into the payload dictionary
requests.post(url, json=payload, verify="myShnazzyCertificate.pem")

Server-side:

payload = flask.request.json

# tag and nonce are unpacked from the above payload dictionary and then
tag        = base64.b64decode(tag)                                              
nonce      = base64.b64decode(nonce)

# ... tag and nonce are used successfully to decrypt data

answered Nov 6, 2024 at 1:27

Ryan Farber

4021 gold badge5 silver badges16 bronze badges

Collectives™ on Stack Overflow

Python urldecode urlencoded bytecode data

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related