0

I have a Java code given to me by a vendor where we generate signature of the payload and send the signature along with the payload in the request. The signature is the same for the same payload no matter how many times the code is run. I converted the same to Golang as well. The problem is I am unable to replicate the same in Python.

Basically the algorithm is:

  1. Generate SHA-1 digest of the payload
  2. Base16 encode the generated SHA-1 digest
  3. Encrypt the encoded digest with RSAPrivateKey
  4. Base16 encode the encrypted digest

My java snippet:

private static String encryptDigest(String raw, String privateKeyString) throws UnsupportedEncodingException, BadPaddingException, IllegalBlockSizeException, InvalidKeyException, NoSuchPaddingException, NoSuchAlgorithmException {
        Key privatekey = null;
        StringReader reader = new StringReader(privateKeyString);
        try {
            PEMReader pemReader = new PEMReader(reader);
            KeyPair keyPair = (KeyPair) pemReader.readObject();
            privateKey = keyPair.getPrivate();
            pemReader.close();
        } catch (IOException i) {
            log.error("Error while initializing Private Key", i);
        }

        MessageDigest md = MessageDigest.getInstance("SHA-1");
        byte[] digest =  md.digest(raw.getBytes());
        byte[] encodedBase16 = Hex.encode(digest);
        String encodedDigest = new String(encodedBase16);
        String strEncrypted = "";
        Cipher cipher = Cipher.getInstance("RSA/ECB/PKCS1PADDING");
        cipher.init(Cipher.ENCRYPT_MODE, privatekey);
        byte[] encrypted = cipher.doFinal(encodedDigest);
        byte[] encoded = Hex.encode(encrypted);
        strEncrypted = new String(encoded);
        return strEncrypted;
    }

My Python snippet:

import hashlib

from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric import padding
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.asymmetric import rsa
import binascii
import os

private_key_pem = """-----BEGIN PRIVATE KEY-----
...
-----END PRIVATE KEY-----"""

def get_signature(payload):
    """Replicates the Go code's signature generation."""

    # 1. Load the private key
    private_key = serialization.load_pem_private_key(
        private_key_pem.encode("utf-8"),
        password=None,
        backend=default_backend()
    )

    private_numbers = private_key.private_numbers()
    public_numbers = private_key.public_key().public_numbers()

    # Print key parameters to compare with OpenSSL & Java
    print("Modulus (n):", hex(public_numbers.n))
    print("Private Exponent (d):", hex(private_numbers.d))
    print("Public Exponent (e):", hex(public_numbers.e)) # Remove '0x' prefix

    # 2. Payload processing
    escaped_payload = payload.replace("\n", "").replace(" ", "").replace('<?xmlversion="1.0"encoding="UTF-8"?>', "")

    # 3. SHA-1 hashing
    sha1_hash = hashlib.sha1(escaped_payload.encode("utf-8")).digest()
    sha1_hash_hex = binascii.hexlify(sha1_hash).decode("utf-8")
    sha1_hash_bytes = binascii.unhexlify(sha1_hash_hex)

    # 4. RSA signing (crucial part)
    signature = private_key.sign(
        sha1_hash_bytes,
        padding.PKCS1v15(),
        hashes.SHA1(),
    )

    # 5. Hex encoding
    hex_signature = binascii.hexlify(signature).decode("utf-8")

    return hex_signature

print(get_signature('helloworld'))

Debugging Steps taken by me to verify:

  1. SHA-1 hash is matching in Java and Python
  2. hex of SHA-1 hash is matching in Java and Python
  3. the private key is read properly and matching. I have matched the modulus, private and public exponent in Java and Python.

From what I got from LLM's and the web is there are subtle difference in how PKCS#1 v1.5 padding is appled in Java and Python. But I don't think that is the problem. Can anyone point me where I am going wrong.

sample private key:

-----BEGIN RSA PRIVATE KEY-----
MIIBOQIBAAJAXWRPQyGlEY+SXz8Uslhe+MLjTgWd8lf/nA0hgCm9JFKC1tq1S73c
Q9naClNXsMqY7pwPt1bSY8jYRqHHbdoUvwIDAQABAkAfJkz1pCwtfkig8iZSEf2j
VUWBiYgUA9vizdJlsAZBLceLrdk8RZF2YOYCWHrpUtZVea37dzZJe99Dr53K0UZx
AiEAtyHQBGoCVHfzPM//a+4tv2ba3tx9at+3uzGR86YNMzcCIQCCjWHcLW/+sQTW
OXeXRrtxqHPp28ir8AVYuNX0nT1+uQIgJm158PMtufvRlpkux78a6mby1oD98Ecx
jp5AOhhF/NECICyHsQN69CJ5mt6/R01wMOt5u9/eubn76rbyhPgk0h7xAiEAjn6m
EmLwkIYD9VnZfp9+2UoWSh0qZiTIHyNwFpJH78o=
-----END RSA PRIVATE KEY-----

Signature generated by java

01540a01e7c372f4d1395221ec90f68a0f4dbc123af9d032b768fd141c0b0a6420e0dcf903739dd2729cfdbf81bcd9512cc39ad4bd26239eab23069fdaf4e6fe

Signature generated by python

252a29b2d5459f4506ab3bb143f24adc60c41a10afb6d7557a1c98cfc244a002eb1490d84780d40233420ef1bc83e005ab7cdb390809c06a757bd84dd790cb2b
11
  • 5
    The Java code performs a low level signing that differs from the RFC8017 standard: Only the message hash is signed, but not the DER encoding of the DigestInfo value. The Python code, on the other hand, signs according to the standard, but hashes twice. This creates different PKCS#1 v1.5 signatures. Commented Mar 27 at 8:49
  • 3
    Another reason for different signatures can be different messages. To rule this out, it is best to compare the hex encoded messages in both codes. These must be completely identical. A single different byte leads to a completely different hash and a completely different signature. Post sample data: Private test key, message, expected signature, generated signature. Commented Mar 27 at 8:50
  • 1
    Adding to @Topaco comments, string encoding handling could be different in both languages. Be sure to compare the hex representation of the bytes before hashing. Usually hashing/encrypting strings can led to differences. Commented Mar 27 at 13:28
  • @Topaco i have added the private key , message , expected signature is the on in java and the generated signatueres are in the one in python Commented Mar 27 at 14:00
  • Your post is inconsistent. The signature 0x01540a... you posted cannot have been generated with the Java code you posted. The Java code generates the SHA-1 hash and signs it. The hash is not hex encoded before signing. On the other hand, your description states that the hash is hex encoded (step 2), and indeed the signature 0x01540a... can only be reproduced with the hex encoded hash. What is correct now? Commented Mar 27 at 14:38

1 Answer 1

-2

I finally got it working by following this comment https://stackoverflow.com/a/43684021/24823862

below is the code for anyone who wants to use it


from M2Crypto import RSA
import binascii
import hashlib
import base64
# Data to hash
data = 'helloworld'
# Compute SHA-1 hash
sha1_hash = hashlib.sha1(data.encode('utf-8')).digest()
# Convert SHA-1 hash to hex string
sha1_hash_str = binascii.hexlify(sha1_hash).decode('utf-8')
# Read private key
private_key = RSA.load_key('key.pem')
# Encrypt the SHA-1 hash string using private key
ciphertext = private_key.private_encrypt(sha1_hash_str.encode('utf-8'), RSA.pkcs1_padding)
#encrypted_message = str(base64.b64encode(ciphertext), 'utf8')
encrypted_message = binascii.hexlify(ciphertext).decode('utf-8')
print(encrypted_message)

generated hash

01540a01e7c372f4d1395221ec90f68a0f4dbc123af9d032b768fd141c0b0a6420e0dcf903739dd2729cfdbf81bcd9512cc39ad4bd26239eab23069fdaf4e6fe

I got M2 crypto running in an ubuntu v22.04.3 LTS machine with python v3.10.12 and followed these instruction

M2 crypto repo - https://gitlab.com/m2crypto/m2crypto/-/tree/master

instructions to install - https://gitlab.com/m2crypto/m2crypto/-/blob/master/INSTALL.rst

Attaching two useful stackoverflow posts that helped me

Python to Java encryption (RSA)

Implementing RSA/pkcs1_padding in Python 3.X

Sign up to request clarification or add additional context in comments.

4 Comments

M2 relies on a very old, badly designed C-library that wasn't maintained for many years. I'm glad you found a solution for your problem, but this code should not be used by anyone.
Note that the Java code may break if any other RSA provider is being used. It requires for the implementation to magically switch to PKCS#1 padding for signature generation which is different from PKCS#1 padding for encryption. So if your application suddenly stops working for you, remember that this code just helped you should you or the next dev in the foot.
understood Maarten Bodewes
Regarding the outdated M2Crypto library: You can also implement the signing logic used in the Java code with the more modern PyCryptodome or pyca/cryptography, see e.g. this PyCryptodome solution with customizedSign().

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.