Support OpenSSL backend and implement message digest - #1492
Support OpenSSL backend and implement message digest#1492KostasTsiounis wants to merge 5 commits into
Conversation
ff3a354 to
26a5ca2
Compare
|
I know we talked about contexts and one for FIPS and one for non-FIPS. Could we use the Property query instead? Does using this have any performance ramifications? |
04bc9c2 to
d4a4c94
Compare
1b76bd7 to
31add61
Compare
8920c23 to
0e09aab
Compare
c43bb68 to
117b14c
Compare
| #include "Utils.h" | ||
| #include <string.h> | ||
|
|
||
| int debug = 0; // FIXME |
There was a problem hiding this comment.
All gslogMessage calls always print to stderr regardless of the flag value
There was a problem hiding this comment.
This is following the same pattern that all OCK native code has. Do you think we should change this to have different levels of logging? We could open an issue to do this everywhere if we want.
dea939a to
1bb6a35
Compare
1bb6a35 to
555ccec
Compare
| * Signature: (JJI)V | ||
| */ | ||
| JNIEXPORT void JNICALL | ||
| Java_com_ibm_crypto_plus_provider_openssl_NativeOpenSSLImplementation_DIGEST_1digest_1and_1reset__JJI( |
There was a problem hiding this comment.
According to the parameters, this should be _JJJI
There was a problem hiding this comment.
Updated comment and method signature.
| static private ConcurrentLinkedQueueLong contexts[]; | ||
|
|
||
| static private int runtimeContextNum[]; | ||
| private static final Map<NativeInterface, Integer[]> runtimeContextNumPerBackend = new HashMap<>(); |
There was a problem hiding this comment.
I have a concern here by using two hashmaps. If I understand correctly, the backend cache state is published through two separate maps. Another thread can observe contextsPerBackend initialized before the corresponding runtimeContextNumPerBackend entry is published, skip initialization, and later dereference a null runtimeContextNum.
There was a problem hiding this comment.
I'm not sure I follow the train of thought here. Are you talking about the static class variables? These will be assigned to a new hashmap during the class initialization. When a thread actually attempts to get a context, it will go through the double check and synchronization lock in lines 75-78. The context queue per backend is also a concurrent one.
Did you mean something else that I missed here?
There was a problem hiding this comment.
Yes, I’m referring to the static per-backend maps, but the issue is not the initialization of the maps themselves or the ConcurrentLinkedQueue.
Consider two threads entering getContext() for a backend that has not been initialized yet. Both threads can initially read contexts == null and runtimeContextNum == null. Thread A acquires the lock first, initializes both entries, and releases the lock. Thread B then acquires the lock and re-reads only contextsPerBackend. Since contexts is now non-null, it skips the initialization block. However, Thread B never re-reads runtimeContextNumPerBackend, so its local runtimeContextNum variable is still null from the read before acquiring the lock.
There was a problem hiding this comment.
Maybe ConcurrentHashMap instead of Map ?
There was a problem hiding this comment.
I make sure to set runtimeContextNum in the synchronized block if contexts has already been set.
| goto cleanup; | ||
| } | ||
|
|
||
| rc = EVP_DigestFinal_ex(mdCtx, digestBytesNative, (unsigned int *)&digestLen); |
There was a problem hiding this comment.
Is there any reason why we call EVP_DigestFinal_ex twice? If you want to get the output buffer size, i believe you can do something like
int digestLen = EVP_MD_CTX_get_size(mdCtx);
digestBytes = (*env)->NewByteArray(env, digestLen);
unsigned int actualLen = 0;
EVP_DigestFinal_ex(
mdCtx,
digestBytesNative,
&actualLen);
There was a problem hiding this comment.
Changed to suggested API. It looks like EVP_DigestFinal_ex doesn't guarantee proper size.
| Java_com_ibm_crypto_plus_provider_openssl_NativeOpenSSLImplementation_DIGEST_1update( | ||
| JNIEnv *env, jclass thisObj, jlong osslContextId, jlong digestId, | ||
| jbyteArray data, jint offset, jint dataLen) { | ||
| //static const char *functionName = "NativeOSSLImplementation.DIGEST_update"; |
| if (digestLen < 0) { | ||
| throwOSSLException(env, 0, "DIGEST_digest_and_reset_internal: The specified data length is negative"); | ||
| return 0; | ||
| } |
There was a problem hiding this comment.
Perhaps we can skip this check since a unsigned int could not be < 0
| void osslCheckStatus(void) { | ||
| unsigned long errCode; | ||
|
|
||
| while ((errCode = ERR_get_error()) == 1) { |
There was a problem hiding this comment.
Are we sure all errors return as a value of 1 ? The OpenSSL doc for this API ERR_get_error indicates a value of 0 if there are no errors left to read from the queue.
while ((errCode = ERR_get_error()) != 0)
There was a problem hiding this comment.
Changed to suggestion.
| static char printBuffer[4096]; | ||
|
|
||
| va_start(formatArgs, formatString); | ||
| charsPrinted = vsprintf(printBuffer, formatString, formatArgs); |
There was a problem hiding this comment.
Could we use vsnprintf instead of vsprintf in all our new openssl code? If formatString were large here or was missing a null terminator I think it would overlay printBuffer. In general all our new code we should try to make as bounded as possible. Something like the following in various places:
vsnprintf(printBuffer, sizeof(printBuffer), formatString, formatArgs);
There was a problem hiding this comment.
Switched to vsnprintf.
| static private ConcurrentLinkedQueueLong contexts[]; | ||
|
|
||
| static private int runtimeContextNum[]; | ||
| private static final Map<NativeInterface, Integer[]> runtimeContextNumPerBackend = new HashMap<>(); |
There was a problem hiding this comment.
Maybe ConcurrentHashMap instead of Map ?
| Java_com_ibm_crypto_plus_provider_openssl_NativeOpenSSLImplementation_DIGEST_1digest_1and_1reset__JJI( | ||
| JNIEnv *env, jclass thisObj, jlong osslContextId, jlong digestId, | ||
| jlong digestBytes, jint length) { | ||
| //static const char *functionName = "NativeOSSLImplementation.DIGEST_digest_and_reset"; |
There was a problem hiding this comment.
Any reason to keep these method names through the code? If we are going to use them then we should add debugging pathing, if not i think we could eliminate these. We could always later copy the value from the comment for the method for the Method: name such that the value would not be lost.
There was a problem hiding this comment.
Followed the same pattern as OCK initially, but I removed them now.
| @Override | ||
| public String getLibraryBuildDate() { | ||
| if (libraryBuildDate == unobtainedValue) { | ||
| libraryBuildDate = NativeOpenSSLImplementation.getLibraryBuildDate();; |
| // value has not yet been obtained. We do this because some values | ||
| // may be null and we only want to query the value one time. | ||
| // | ||
| private static String libraryBuildDate = unobtainedValue; |
There was a problem hiding this comment.
Does this have to be static ? I could see how there are two threads trying to set libraryBuildDate below might cause some issues?
There was a problem hiding this comment.
They would just set it to the same value. I don't think there is a need to keep one per instance.
| EVP_MD_free(md); | ||
| md = NULL; | ||
|
|
||
| (*env)->ReleaseStringUTFChars(env, digestAlgo, digestAlgoChars); |
There was a problem hiding this comment.
We might need a null check on digestAlgoChars here? If an exception is thrown prior to line 41 then digestAlgoChars would be null.
There was a problem hiding this comment.
If an exception is thrown that early, we don't go to cleanup. We just return 0.
| public static NativeOpenSSLAdapterNonFIPS getInstance() { | ||
| if (instance == null) { | ||
| instance = new NativeOpenSSLAdapterNonFIPS(); | ||
| } |
There was a problem hiding this comment.
Is it guaranteed that NativeOpenSSLAdapterNonFIPS.getInstance() returns a single instance under concurrent first access? getInstance() itself is not synchronized and its instance field is not volatile, so two threads may construct different adapter instances.
This seems particularly relevant now that Digest uses the NativeInterface instance itself as the key for contextsPerBackend. Two adapter instances representing the same OpenSSL backend would therefore create separate digest caches.
There was a problem hiding this comment.
You are right. Changed to do a double check with synchronization in between.
With this change, all appropriate code is added to support the use of OpenSSL as a backend. That includes: - Main Java class to load the libraries required and offer native methods. - Adapter Java classes to smoothly integrate without changes with existing code. - Makefiles to compile and link required C code to use OpenSSL. - Additional test tags and general test setup to allow the execution of appropriate tests with the new backend. Additionally, the first set of algorithms, namely the message digests, are implemented. The aforementioned tags are, also, added to the corresponding tests. Signed-off-by: Kostas Tsiounis <kostas.tsiounis@ibm.com>
521f2a4 to
9db9a0b
Compare
fdd6070 to
f01a83d
Compare
With this change, all appropriate code is added to support the use of
OpenSSLas a backend. That includes:OpenSSL.Additionally, the first set of algorithms, namely the message digests, are implemented. The aforementioned tags are, also, added to the corresponding tests.
Signed-off-by: Kostas Tsiounis kostas.tsiounis@ibm.com