1

I generated RSA private key and public key as below,

openssl genpkey -algorithm RSA -out key.pem -pkeyopt rsa_keygen_bits:2048
openssl rsa -pubout -in pri.key -out pub.key

And encrypted text file as below,

openssl pkeyutl -encrypt -pubin -inkey ~/pub.key -in ~/1.txt -out ~/1e.txt

Then I wrote below program to decrypt the encryted file. However, it seemed that decrypt didn't work as expected.

#include <openssl/evp.h>
#include <openssl/rsa.h>
#include <openssl/pem.h>
#include <openssl/err.h>
#include <openssl/conf.h>
#include <iostream>

using namespace std;

void
cleanup()
{
    EVP_cleanup();
    CRYPTO_cleanup_all_ex_data();
    ERR_free_strings();
}

int
main(int argc, char** argv)
{
    ERR_load_crypto_strings();
    OpenSSL_add_all_algorithms();
    OPENSSL_config(nullptr);

    cout<<"Initialize crypto library done"<<endl;

    EVP_PKEY * key = EVP_PKEY_new();
    if (key == nullptr) {
        cout<<"Failed to contruct new key"<<endl;
        return 1;
    }
    FILE * fpri = nullptr;
    fpri = fopen("/home/stack/pri.key", "r");
    if (fpri == nullptr) {
        cout<<"Failed to load private key"<<endl;
        return 1;
    }
    key = PEM_read_PrivateKey(fpri, &key, nullptr, nullptr);
    if (key == nullptr) {
        std::cout<<"Read private key failed"<<endl;
        return 1;
    }
    cout<<"load private key successfully"<<endl;
    EVP_PKEY_CTX *ctx = nullptr;
    ctx = EVP_PKEY_CTX_new(key, nullptr);
    EVP_PKEY_decrypt_init(ctx);
    EVP_PKEY_CTX_set_rsa_padding(ctx, RSA_NO_PADDING);

    size_t outlen = 0, inlen = 0;
    unsigned char * out = nullptr, * in = nullptr;

    char buf[1024];
    FILE * fe = nullptr;
    fe = fopen("/home/stack/1e.txt", "r");
    size_t len = fread(buf, 1, sizeof(buf),  fe);
    cout<<"data input length is "<<len<<endl;
    EVP_PKEY_decrypt(ctx, NULL, &outlen, in, inlen);
    cout<<"outlen is "<<outlen<<endl;

    out = (unsigned char*)OPENSSL_malloc(outlen);
    EVP_PKEY_decrypt(ctx, out, &outlen, in, inlen);
    cout<<"decrypted data "<<out<<endl;
    cleanup();

    return 0;

}

When executing the code, the result is as below,

[stack@agent ~]$ ./test
Initialize crypto library done
load private key successfully
data input length is 256
outlen is 256
decrypted data

It seemed the decrypted data length was not correct and not printable.

When I commented out the instruction "EVP_PKEY_CTX_set_rsa_padding(ctx, RSA_NO_PADDING);", it worked well.

I also tried with RSA_PKCS1_OAEP_PADDING, it didn't work either. If RSA PADDING is not set, it worked.

My question is as below,

  1. Which padding is used in following command?

    openssl pkeyutl -encrypt -pubin -inkey ~/pub.key -in ~/1.txt -out ~/1e.txt
    
  2. Is padding necessary for RSA encrypt/decrypt? If so, how could I apply the padding mechanism?

4
  • The documentation doesn't say which padding is used by default. I would think it's PKCS#1 padding. Commented Mar 23, 2017 at 7:03
  • @ArtjomB. I tried to use PKCS#1 padding to decrypt firstly, and it didn't work. Then I guessed that no padding is used and use RSA_NO_PADDING to decrypt, it didn't work either. But after I commented the instruction for setting rsa_padding, it worked. Although my code works now, I still don't quite understand why. Commented Mar 23, 2017 at 7:56
  • @ArtjomB. I checked openssl source code. I understand it now, PKCS#1 is default padding mode. Commented Mar 23, 2017 at 9:00
  • If you looked into the code, then you can probably provide a more extensive answer than the currently accepted one. Think about it. Commented Mar 23, 2017 at 18:56

1 Answer 1

0

You should use EVP_PKEY_CTX_set_rsa_padding if you use not default padding in openssl pkeyutl encryption. See openssl pkeyutl documentation for details about -rsa_padding_mode.

Sign up to request clarification or add additional context in comments.

1 Comment

Thanks for your explanation. I checked source code, PKCS#1 is default padding mode.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.