EVALUATION
The main problem was that the 128 threads were creating objects quickly, and only one Finalizer thread was being run. As a result, there were a large number of native-heap backed Java objects waiting for finalization, and thus "leaking" memory.
Also, the JSSE ciphers were never calling Cipher.doFinal(), and thus the Cipher objects had to cancel their PKCS11 backed cipher operations, which was also expensive and adding to the finalizer backlog.
We converted the P11Keys to use WeakReferences, plus added the doFinal in JSSE, and in the P11SecretKeyFactory to remove the final cancellation, and that took care of the finalization backlog. Still showing a small amount of leaking, which is under investigation. I put in some code to show PKCS11 native malloc/frees, and all are matching up. Whereever this new leak is, it's not from the PKCS11 native calls.
|
EVALUATION
Much of the IAIK code does not check for (malloc() == null) problems, which will lead to SIGSEGV's if memory is out. This needs to be fixed, and will be a fair amount of work.
In the first example: hs_err_pid6293.log, this clearly shows a null pointer being dereferenced following a call to wrapper_PKCS11_C_1EncryptUpdate, and a SIGSEGV being returned.
This is not the root cause of the memory leak, but definitely a problem which needs to be fixed.
|
Java