'Encrypted message length is invalid.' using node-forge to decrypt the message encrypted from jsencrypt

Question

currently using jsencrypt and node-forge for decrypt and encrypt the message using RSA, while jsencrypt is used in frontend and node-forge in backend.

import { JSEncrypt } from 'jsencrypt'
import * as forge from 'node-forge';

const message = 'data....'
const publicKey = '-----BEGIN PUBLIC KEY-----\nMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCqM+l9ZWy1Frt6felFFLmfZNls\nVbU1dKpF8Rx83FtKCsztO5k/iV5N9BbfHFUg9Y40b/EK2j/BPc1xlLYAHMXn6563\nXCwZ4IuCxvfOwz9qT9gkKBxkI5b0rnikkSWTGlJEk2PdZ7Plc73Fa+bx3PvuKvMd\ncKWvd80+vt9+b/7hrwIDAQAB\n-----END PUBLIC KEY-----'

const publicK = forge.pki.publicKeyFromPem(publicKey)
const encrypted = publicK.encrypt( message, 'RSA-OAEP')
console.log('encrpted：', encrypted) 

const privateKey = '-----BEGIN PRIVATE KEY-----\nMIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBAKoz6X1lbLUWu3p9\n6UUUuZ9k2WxVtTV0qkXxHHzcW0oKzO07mT+JXk30Ft8cVSD1jjRv8QraP8E9zXGU\ntgAcxefrnrdcLBngi4LG987DP2pP2CQoHGQjlvSueKSRJZMaUkSTY91ns+VzvcVr\n5vHc++4q8x1wpa93zT6+335v/uGvAgMBAAECgYArxUnou6qnL39rUvIol9ncyfy4\nRZpicuxPLGCdI7Y+ZmSpJciVdGhSN9Gh8xFZdozpo1gj6Fi5A4HQEeR0RvIF9Rgh\nERblj1rRWqxPcsIddOO9VaknQPICWKqEW9+E1bEcyNUblCHA4LGyQwmuEFUb/Tkj\nxAghIHuEBCe0GFiVwQJBAN5i5QSoOIpdFHA0c981E4VhHc/muXwjx1HfE1pcuuFb\nTy3OwEoZdFp3LIjBnBkPRneLTNjo5WTIwrmfsy6VDF8CQQDD7c6d/nKiJwIESlr+\n/idqXAPNR/iS1YX3Nqtk9jgrgf5zULHr2nbk7MDas5S9Z9XPdUmxtnP44dhoGvDk\nzyyxAkB7XBxyQuZqSkvGGjKUhJq5iC/DXddSd35fegEARSQdUktPu7qK4Cfc7vKz\nQcLXW9PZCFqukDJ/f6YU1fPNSTy9AkADQ78hms/GK+g4shR6EzoM56OYlA5sQ+qL\nh/mrIP8mmm/m8/1C9MzuW5OLEVr1HPnPDyE/OM8N4pV8hpZk+Z7BAkEAzaFstazA\nxLzZOBWhvOzzo722glZ7HVezhMocLu7Y3EOXP/nbx09JpU3U7Egp5UVp0aiknh/Q\nez4Cc4ksMedxdA==\n-----END PRIVATE KEY-----\n'
const privateK = forge.pki.privateKeyFromPem(privateKey)

const decrypted = privateK.decrypt(encrypted, 'RSA-OAEP')
console.log('original：', decodeURIComponent(decrypted))

this worked.

then trying with jsencrypt with same pub/pri key and message.

  const encrypt = new JSEncrypt();
  encrypt.setPublicKey(publicKey);
  let encrypedQuery = encrypt.encrypt( message );
  console.log( encrypedQuery );

  try{
    const privateK = forge.pki.privateKeyFromPem(privateKey)
    const decrypted2 = privateK.decrypt(encrypedQuery)
    console.log('original：：', decodeURIComponent(decrypted2)) 
  }catch(err){
    console.log(err);
    
  }

this outputs an error saying 'Encrypted message length is invalid.'

jsencrypt, designed for browser use, returns (and takes) ciphertext as base64, while forge uses bytes. atob of jsencrypt result will be correct size, but still won't interoperate because jsencrypt, built on ancient code, does only 'classic' PKCS1v1.5 padding, not OAEP. If you specify or default 'RSAES-PKCS1-V1_5' in forge it should decrypt, but your application may be vulnerable to Bleichenbacher attacks, while 1024-bit RSA may soon be broken outright, both of which are offtopic here; see crypto.SX and security.SX. — dave_thompson_085
– dave_thompson_085, Commented Nov 28, 2024 at 7:26
@dave_thompson_085 - node-forge applies PKCS#1 v1.5 by default, so the problem in the second code snippet is only an encoding issue. — Topaco
– Topaco, Commented Nov 28, 2024 at 11:18

Topaco · Accepted Answer · 2024-11-28 14:14:26Z

The problem in the second code snippet is an encoding issue that arises because the JSEncrypt side returns a Base64 encoded ciphertext, while node-forge requires a binary/latin1 string. The issue can be fixed with forge.util.decode64() (or atob()):

const publicKey = '-----BEGIN PUBLIC KEY-----\nMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCqM+l9ZWy1Frt6felFFLmfZNls\nVbU1dKpF8Rx83FtKCsztO5k/iV5N9BbfHFUg9Y40b/EK2j/BPc1xlLYAHMXn6563\nXCwZ4IuCxvfOwz9qT9gkKBxkI5b0rnikkSWTGlJEk2PdZ7Plc73Fa+bx3PvuKvMd\ncKWvd80+vt9+b/7hrwIDAQAB\n-----END PUBLIC KEY-----'
const privateKey = '-----BEGIN PRIVATE KEY-----\nMIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBAKoz6X1lbLUWu3p9\n6UUUuZ9k2WxVtTV0qkXxHHzcW0oKzO07mT+JXk30Ft8cVSD1jjRv8QraP8E9zXGU\ntgAcxefrnrdcLBngi4LG987DP2pP2CQoHGQjlvSueKSRJZMaUkSTY91ns+VzvcVr\n5vHc++4q8x1wpa93zT6+335v/uGvAgMBAAECgYArxUnou6qnL39rUvIol9ncyfy4\nRZpicuxPLGCdI7Y+ZmSpJciVdGhSN9Gh8xFZdozpo1gj6Fi5A4HQEeR0RvIF9Rgh\nERblj1rRWqxPcsIddOO9VaknQPICWKqEW9+E1bEcyNUblCHA4LGyQwmuEFUb/Tkj\nxAghIHuEBCe0GFiVwQJBAN5i5QSoOIpdFHA0c981E4VhHc/muXwjx1HfE1pcuuFb\nTy3OwEoZdFp3LIjBnBkPRneLTNjo5WTIwrmfsy6VDF8CQQDD7c6d/nKiJwIESlr+\n/idqXAPNR/iS1YX3Nqtk9jgrgf5zULHr2nbk7MDas5S9Z9XPdUmxtnP44dhoGvDk\nzyyxAkB7XBxyQuZqSkvGGjKUhJq5iC/DXddSd35fegEARSQdUktPu7qK4Cfc7vKz\nQcLXW9PZCFqukDJ/f6YU1fPNSTy9AkADQ78hms/GK+g4shR6EzoM56OYlA5sQ+qL\nh/mrIP8mmm/m8/1C9MzuW5OLEVr1HPnPDyE/OM8N4pV8hpZk+Z7BAkEAzaFstazA\nxLzZOBWhvOzzo722glZ7HVezhMocLu7Y3EOXP/nbx09JpU3U7Egp5UVp0aiknh/Q\nez4Cc4ksMedxdA==\n-----END PRIVATE KEY-----\n'
const message = 'data....'

const encrypt = new JSEncrypt();
encrypt.setPublicKey(publicKey);
let encryptedQuery = encrypt.encrypt(message);
console.log(encryptedQuery);

try {
    const privateK = forge.pki.privateKeyFromPem(privateKey);
    const decrypted2 = privateK.decrypt(forge.util.decode64(encryptedQuery)); // fix
    console.log('original：：', decodeURIComponent(decrypted2)); 
} catch(err){
    console.log(err);
}

<script src="https://cdnjs.cloudflare.com/ajax/libs/forge/1.3.1/forge.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/jsencrypt/3.3.2/jsencrypt.min.js"></script>

Note that the first code snippet uses OAEP as padding (explicitly specified), while the second code snippet applies PKCS#1 v1.5 padding (JSEncrypt exclusively uses PKCS#1 v1.5, node-forge applies it by default).
Since the padding is consistent within the same code snippet, decryption works. However, it would not be possible to decrypt a ciphertext generated with the first code snippet with the second or vice versa.

JSEncrypt with OAEP:

Although node-forge supports PKCS#1 v1.5 (by default) and OAEP as padding, the padding of the regular JSEncrypt library is PKCS#1 v1.5 and cannot be changed. Therefore, only PKCS#1 v1.5 can be used for interoperation between the two libraries.

As mentioned in the comment, the older PKCS#1 v1.5 padding is vulnerable to certain attacks and OAEP is the more secure alternative.
If you want to use JSEncrypt with OAEP: There is a JSEncrypt pull request on Github and a fork that extends JSEncrypt with OAEP, which might be an option for you. However, this uses SHA-256 for both the OAEP and MGF1 digests, while node-forge applies SHA-1 by default, but also supports other digests. Since the digest on the JSEncrypt side cannot be changed, the digest on the node-forge side must be changed to SHA-256 for interoperation, e.g.:

const publicKey = '-----BEGIN PUBLIC KEY-----\nMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCqM+l9ZWy1Frt6felFFLmfZNls\nVbU1dKpF8Rx83FtKCsztO5k/iV5N9BbfHFUg9Y40b/EK2j/BPc1xlLYAHMXn6563\nXCwZ4IuCxvfOwz9qT9gkKBxkI5b0rnikkSWTGlJEk2PdZ7Plc73Fa+bx3PvuKvMd\ncKWvd80+vt9+b/7hrwIDAQAB\n-----END PUBLIC KEY-----'
const privateKey = '-----BEGIN PRIVATE KEY-----\nMIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBAKoz6X1lbLUWu3p9\n6UUUuZ9k2WxVtTV0qkXxHHzcW0oKzO07mT+JXk30Ft8cVSD1jjRv8QraP8E9zXGU\ntgAcxefrnrdcLBngi4LG987DP2pP2CQoHGQjlvSueKSRJZMaUkSTY91ns+VzvcVr\n5vHc++4q8x1wpa93zT6+335v/uGvAgMBAAECgYArxUnou6qnL39rUvIol9ncyfy4\nRZpicuxPLGCdI7Y+ZmSpJciVdGhSN9Gh8xFZdozpo1gj6Fi5A4HQEeR0RvIF9Rgh\nERblj1rRWqxPcsIddOO9VaknQPICWKqEW9+E1bEcyNUblCHA4LGyQwmuEFUb/Tkj\nxAghIHuEBCe0GFiVwQJBAN5i5QSoOIpdFHA0c981E4VhHc/muXwjx1HfE1pcuuFb\nTy3OwEoZdFp3LIjBnBkPRneLTNjo5WTIwrmfsy6VDF8CQQDD7c6d/nKiJwIESlr+\n/idqXAPNR/iS1YX3Nqtk9jgrgf5zULHr2nbk7MDas5S9Z9XPdUmxtnP44dhoGvDk\nzyyxAkB7XBxyQuZqSkvGGjKUhJq5iC/DXddSd35fegEARSQdUktPu7qK4Cfc7vKz\nQcLXW9PZCFqukDJ/f6YU1fPNSTy9AkADQ78hms/GK+g4shR6EzoM56OYlA5sQ+qL\nh/mrIP8mmm/m8/1C9MzuW5OLEVr1HPnPDyE/OM8N4pV8hpZk+Z7BAkEAzaFstazA\nxLzZOBWhvOzzo722glZ7HVezhMocLu7Y3EOXP/nbx09JpU3U7Egp5UVp0aiknh/Q\nez4Cc4ksMedxdA==\n-----END PRIVATE KEY-----\n'
const message = 'data....'

const encrypt = new JSEncrypt();
encrypt.setPublicKey(publicKey);
let encryptedQuery = encrypt.encryptOAEP(message); // OAEP with SHA256
console.log(encryptedQuery);

try {
    const privateK = forge.pki.privateKeyFromPem(privateKey);
    const decrypted2 = privateK.decrypt(forge.util.decode64(encryptedQuery), 'RSA-OAEP', {md: forge.md.sha256.create()}); // OAEP with SHA256
    console.log('original：：', decodeURIComponent(decrypted2)); 
} catch(err){
    console.log(err);
}

<script src="http://cdn.jsdelivr.net/gh/kingller/jsencrypt/bin/jsencrypt.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/forge/1.3.1/forge.min.js"></script>

Collectives™ on Stack Overflow

'Encrypted message length is invalid.' using node-forge to decrypt the message encrypted from jsencrypt

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related