RSA Signing and Encryption with NSS

NSS Technical Note: 7

This technical note explains how to use NSS to perform RSA signing and encryption. The industry standard for RSA signing and encryption is PKCS #1. NSS supports PKCS #1 v1.5. NSS doesn't yet support PKCS #1 v2.0 and v2.1, in particular OAEP, but OAEP support is on our to-do list. Your contribution is welcome.

Data Types

NSS uses the following data types to represent keys:

These data types should be used as if they were opaque structures, that is, they should only be created by some NSS functions and you always pass pointers to these data types to NSS functions and never examine the members of these structures.

The strength of an RSA key pair is measured by the size of its modulus because given the modulus and public exponent, the best known algorithm for computing the private exponent is to factor the modulus. At present 1024 bit and 2048 bit RSA keys are the most common and recommended. To prevent denial-of-service attacks with huge public keys, NSS disallows modulus size greater than 8192 bits.

How are these keys created in NSS? There are a few possibilities.

When the keys are no longer needed, they need to be destroyed.

Functions

RSA signing and encryption functions are provided by two layers of NSS function: the SGN_ and VFY_ functions in cryptohi.h, and the PK11_ functions in pk11pub.h. As a general principle, you should use the highest layer of NSS you can possibly use for what you are trying to accomplish.

For example, if you just need to generate or verify a signature, you can use the SGN_ and VFY_ functions in cryptohi.h.

If you need to interoperate with a protocol that isn't implemented by NSS, then you may need to use the PK11_ functions. (This API pretty much consists of what was needed to implement SSL and S/MIME, plus a few enhancements over the years to support JSS.) When using the PK11_ interfaces, the same principal applies: use the highest available function.

If you are really trying to send a key, you should use PK11_PubWrapSymKey(). For a low level signature, use PK11_Sign(). Both of these functions do the PKCS #1 wrapping of the data. PK11_Sign does not do the BER encoding of the hash (as is done in SGN_ functions).

If you are trying to just send data, use PK11_PubEncryptPKCS1.

PK11_PubEncryptRaw is the lowest level function. It takes a modulus size data and does a raw RSA operation on the data. It's used to support SSL2, which modifies the key encoding to include the SSL version number.

PKCS #1 v1.5 Block Formatting

Question:

In PKCS #1 v1.5 (Section 8.1 Encryption-block formatting) and v2.1 (Section 7.2.1 Encryption operation), PKCS1 v1.5 padding is described like this:

00 || 02 || PS || 00 || M

but in PKCS #1 v2.0 (Section 9.1.2.1 Encoding operation, Step 3) and on the W3C web site (http://www.w3.org/TR/xmlenc-core/#rsa-1_5), PKCS1 v1.5 padding is described like this:

02 || PS || 00 || M

00 at the beginning is missing. Why?

Answer:

The version without the initial 00 says :

"PS is a string of strong pseudo-random octets [RANDOM] [...] long enough that the value of the quantity being CRYPTed is one octet shorter than the RSA modulus"


The version with the initial 00 instead says to pad to the same length as the RSA modulus.

"The same length as the RSA modulus with an initial octet of 0" and "one octet shorter without that initial octet" are exactly the same thing because the formatted block is treated as a big-endian big integer by the RSA algorithm. The leading 00 octet is simply eight most significant 0 bits. For example, 0x00123456 is equal to 0x123456.

Perhaps this change made in PKCS #1 v2.0 confused many people, so it was reversed in v2.1.

Sample Code

References