[RFC][RESEND] New hash type needed for RSA_sign()

[Arnaud EBALARD]

Hi,                

This is a resend [1], please redirect me if I am off-topic.

while modifying some IKE code to use OpenSSL in order to have RSA
signature done by engines (support via opensc on Linux, more
specifically eGate and eToken PRO devices), I encountered the following
issue.

RSA_sign() has a 'type' argument that describes the hash that was used
for generating the data. It is used to add digest info before padding
and private key encryption occur to complete the signature. There is a
specific case in the code, namely for SSL which does not use a digest
info but a fixed length 36 bytes hash value (concatenation of md5 and
sha1 hashes).

IKEv1 has the same kind of requirement. RFC 2409 has the following in
Section 5.1:

   Since the hash algorithm used is already known there is no need to
   encode its OID into the signature. In addition, there is no binding
   between the OIDs used for RSA signatures in PKCS #1 and those used in
   this document. Therefore, RSA signatures MUST be encoded as a private
   key encryption in PKCS #1 format and not as a signature in PKCS #1
   format (which includes the OID of the hash algorithm). DSS signatures
   MUST be encoded as r followed by s.

Note that I cannot call RSA_private_encrypt() as a workaround because
the devices do not support it (even if on tokens, I have added specific
keys that allow decryption) and it does not seem the expected way to do
it.

At the moment, I am using NID_md5_sha1 type for RSA_sign() and have
temporarily deactivated the length check (36 bytes) in libp11 code but
this is also a temporary workaround. What would be the correct way to
handle that issue? Initially, I wanted to add a new NID_IKE_digest type
that would make RSA_sign() behave like with NID_md5_sha1 but with a
relaxed check on the length. Because the removal of the length check
basically provides access to private key encryption via signature
primitive, I thought I should ask before.

Comments welcome.

Cheers,

a+

[1]: http://thread.gmane.org/gmane.comp.encryption.openssl.devel/14093

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       openssl-dev@...
Automated List Manager                           majordomo@...

[Dr. Stephen Henson]

On Wed, Jul 16, 2008, Arnaud Ebalard wrote:

> Hi,                
>
> This is a resend [1], please redirect me if I am off-topic.
>
> while modifying some IKE code to use OpenSSL in order to have RSA
> signature done by engines (support via opensc on Linux, more
> specifically eGate and eToken PRO devices), I encountered the following
> issue.
>
> RSA_sign() has a 'type' argument that describes the hash that was used
> for generating the data. It is used to add digest info before padding
> and private key encryption occur to complete the signature. There is a
> specific case in the code, namely for SSL which does not use a digest
> info but a fixed length 36 bytes hash value (concatenation of md5 and
> sha1 hashes).
>
> IKEv1 has the same kind of requirement. RFC 2409 has the following in
> Section 5.1:
>
>    Since the hash algorithm used is already known there is no need to
>    encode its OID into the signature. In addition, there is no binding
>    between the OIDs used for RSA signatures in PKCS #1 and those used in
>    this document. Therefore, RSA signatures MUST be encoded as a private
>    key encryption in PKCS #1 format and not as a signature in PKCS #1
>    format (which includes the OID of the hash algorithm). DSS signatures
>    MUST be encoded as r followed by s.
>
> Note that I cannot call RSA_private_encrypt() as a workaround because
> the devices do not support it (even if on tokens, I have added specific
> keys that allow decryption) and it does not seem the expected way to do
> it.
>
> At the moment, I am using NID_md5_sha1 type for RSA_sign() and have
> temporarily deactivated the length check (36 bytes) in libp11 code but
> this is also a temporary workaround. What would be the correct way to
> handle that issue? Initially, I wanted to add a new NID_IKE_digest type
> that would make RSA_sign() behave like with NID_md5_sha1 but with a
> relaxed check on the length. Because the removal of the length check
> basically provides access to private key encryption via signature
> primitive, I thought I should ask before.
>

Well RSA_private_encrypt() despite its name is a signing operation. There is a
PKCS#11 equivalent (C_Sign and the CKM_RSA_PKCS mechanism) though some tokens
don't support it fully but emulate it in such a way that it only works for a
fixed range of digests.

If a token supports it but the PKCS#11 ENGINE has no equivalent that's an
ENGINE implementation issue.

If the actual token doesn't fully support an RSA_private_encrypt() equivalent
then you are SOL.

Using RSA_sign() with NID_md5_sha1 had the problem that the length is
incorrect which is why you need to disable the length check. This has some
security implications: for example the well publicised exponent 3 issue.

Steve.
--
Dr Stephen N. Henson. Email, S/MIME and PGP keys: see homepage
OpenSSL project core developer and freelance consultant.
Homepage: http://www.drh-consultancy.demon.co.uk
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       openssl-dev@...
Automated List Manager                           majordomo@...

[Arnaud EBALARD]

Hi Stephen,

"Dr. Stephen Henson" <steve@...> writes:

> Well RSA_private_encrypt() despite its name is a signing
> operation. There is a PKCS#11 equivalent (C_Sign and the CKM_RSA_PKCS
> mechanism) though some tokens don't support it fully but emulate it in
> such a way that it only works for a fixed range of digests.

Removing the length check in libp11 code that interfaces with OpenSSL
corrects the issue, which seems to indicate that the hardware has no
problem with the type of hash (i.e. does not verify it). I tested again
with blobs of 4, 33 and 88 bytes with both tokens: no pb.

> If a token supports it but the PKCS#11 ENGINE has no equivalent that's
> an ENGINE implementation issue.

I just took some minutes to look at the code and RSA_sign()
implementation in libp11 (PKCS11_sign() in p11_ops.c) basically makes a
direct PKCS#11 call (C_Sign and CKM_RSA_PKCS) after the hash length
check and OID additions when OpenSSL call RSA_private_encrypt().

In libp11, the implementation of RSA_private_encrypt() is an empty shell
that simply returns "Not supported".

In the end, if I follow your first comment (C_Sign and the CKM_RSA_PKCS
mechanism are the PKCS#11 equivalent of RSA_private_encrypt()), I don't
see why the PKCS#11 call is not in the implementation of
RSA_private_encrypt() in libp11, which would be called by RSA_sign().

I will check that with the libp11 developers and stop the noise on
openssl-dev. I'll made a post here at the end of the story.

> If the actual token doesn't fully support an RSA_private_encrypt()
> equivalent then you are SOL.

there is some hope, I think ;-)

> Using RSA_sign() with NID_md5_sha1 had the problem that the length is
> incorrect which is why you need to disable the length check. This has
> some security implications: for example the well publicised exponent 3
> issue.

That's what I thought.


Thanks for your time and your comments,

Cheers,

a+
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       openssl-dev@...
Automated List Manager                           majordomo@...

[Arnaud EBALARD]

Hi,

arno@... (Arnaud Ebalard) writes:

>> If a token supports it but the PKCS#11 ENGINE has no equivalent that's
>> an ENGINE implementation issue.
>
> I just took some minutes to look at the code and RSA_sign()
> implementation in libp11 (PKCS11_sign() in p11_ops.c) basically makes a
> direct PKCS#11 call (C_Sign and CKM_RSA_PKCS) after the hash length
> check and OID additions when OpenSSL call RSA_private_encrypt().
>
> In libp11, the implementation of RSA_private_encrypt() is an empty shell
> that simply returns "Not supported".
>
> In the end, if I follow your first comment (C_Sign and the CKM_RSA_PKCS
> mechanism are the PKCS#11 equivalent of RSA_private_encrypt()), I don't
> see why the PKCS#11 call is not in the implementation of
> RSA_private_encrypt() in libp11, which would be called by RSA_sign().
>
> I will check that with the libp11 developers and stop the noise on
> openssl-dev. I'll made a post here at the end of the story.

Last week, I submitted a patch to OpenSC developers, which adds support
in libp11 for PKCS11_private_encrypt() (the handler used when OpenSSL
RSA_private_encrypt() is caled). It has been accepted, commited to their
SVN and is now available in the recently released 0.2.4 version.

Cheers,

a+