EVALUATION

The current impl of SunJCE provider will strip off the first byte when the bit length is multiples of 8. However, the impl of SunPKCS11 provider strips off all leading 0s as it tries to match what SunJCE provider does.

According to the comment in SunJCE provider source code, the removal of 1st byte is meant to remove the "sign" byte returned by BigInteger.toByteArray() calls. However, in the same impl class, the checking of output buffer didn't take into account of the "sign" byte, thus leading to ShortBufferException as demonstrated in the com/sun/crypto/provider/KeyAgreement/DHKeyAgreement2.java regression test which allocate an output buffer based on the result returned by the KeyAgreement.generateSecret() call.

As for SunPKCS11 provider, it strips off all leading 0x00 bytes as an attempt to "follow JCE convention" when KeyAgreement.generateSecret() is called. However, in its impl of KeyAgreement.generateSecret(byte[] out, int outOfs), it again errors out if the output buffer 'out' starting offset 'outOfs' cannot hold byte arrays as long as the modulus.

This inconsistency between KeyAgreement.generateSecret() and KeyAgreement.generateSecret(byte[], int) is very confusing and should be straightened out.

The current javadoc of KeyAgreement didn't specify whether the leading 0s should be preserved or not, so it could go either way.

Given that some internet standard such as RFC2631 explicitly mentions that the leading zeros should be preserved so the generated secret occupies as many octets as p.

I think the correct fix is to change the SunJCE and SunPKCS11 provider to return a fixed size secret which is the same size as the modulus. This makes maintaining compatibility across applications easier since the generated secret length will then stay the same as the modulus size.