Java RSA encryption keeps generating output that cannot be decrypted?

Question

I'm trying my darnedest to create a program that will generate a private/public RSA key set and use it to send messages that are secure from end to end. I am using the RSA private/public key pair to securely transmit an AES key, that will be used to send the messages.

When I use a 1024-bit key pair, the encrypted session key is 128 bytes. But attempting to decrypt it with the RSA private key, it complains that it can only decrypt 117 bytes or less.

When I use a 2048-bit key pair, the encrypted session key is 256 bytes (but it must be 245 or less), etc.

The result is always the same: I cannot pass a valid passphrase back and forth. :(

    KeyPairGenerator keyPairGenerator = KeyPairGenerator.getInstance("RSA");
    keyPairGenerator.initialize(1024);
    KeyPair keyPair = keyPairGenerator.generateKeyPair();

    KeyGenerator keyGen = KeyGenerator.getInstance("AES");
    SecretKey sessionKey = keyGen.generateKey();

    // Encrypt the session key with the RSA public key
    Cipher rsaCipher = Cipher.getInstance("RSA");
    rsaCipher.init(Cipher.ENCRYPT_MODE, keyPair.getPublic());
    byte[] encryptedSessionKey = rsaCipher.doFinal(sessionKey.getEncoded());

    // Simulating other end user: Receive encrypted session key,
    // decrypt it using your private key.
    Cipher rsaDecryptCipher = Cipher.getInstance("RSA");
    rsaDecryptCipher.init(Cipher.DECRYPT_MODE, keyPair.getPrivate());
    byte[] decryptedSessionKeyBytes = rsaCipher.doFinal(encryptedSessionKey);
    System.out.println("Decrypted session key bytes");
    System.out.println(decryptedSessionKeyBytes);

Edit: It looks like the key that is generated is being padded at some time or another, but I don't know how to stop or subvert it. I has teh dumb when it comes to encryption... up until now I was relying on openssl_seal.

Small note: printing decryptedSessionKeyBytes directly won't work, you need to convert it to hexadecimals first (or, if you want to do something quickly use, Arrays.toString(byte[]) — Maarten Bodewes
– Maarten Bodewes, Commented Aug 14, 2014 at 17:19
Good point @owlstead, from the beginning I was using something along the lines of (for a one-liner): (new BASE64Encoder()).encode(decryptedSessionKeyBytes) — user1499731
– user1499731, Commented Aug 14, 2014 at 17:23

Maarten Bodewes · Accepted Answer · 2014-08-14 17:36:25Z

3

You are using rsaDecryptCipher it for everything except the actual decryption.

It would be a good idea to always specify the mode of operation and padding mode, e.g. "RSA/ECB/OAEPWithSHA1AndMGF1Padding" for your Cipher instances.

edited Aug 14, 2014 at 17:36

answered Aug 14, 2014 at 17:30

Maarten Bodewes

94.6k15 gold badges169 silver badges289 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Maarten Bodewes Over a year ago

I expect a facepalm :)

user1499731 Over a year ago

I am the king of facepalm.

Collectives™ on Stack Overflow

Java RSA encryption keeps generating output that cannot be decrypted?

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related