Skip to content

Support OpenSSL backend and implement message digest - #1492

Open
KostasTsiounis wants to merge 5 commits into
IBM:mainfrom
KostasTsiounis:openssl_backend
Open

Support OpenSSL backend and implement message digest#1492
KostasTsiounis wants to merge 5 commits into
IBM:mainfrom
KostasTsiounis:openssl_backend

Conversation

@KostasTsiounis

@KostasTsiounis KostasTsiounis commented May 22, 2026

Copy link
Copy Markdown
Member

With this change, all appropriate code is added to support the use of OpenSSL as a backend. That includes:

  • Main Java class to load the libraries required and offer native methods.
  • Adapter Java classes to smoothly integrate without changes with existing code.
  • Makefiles to compile and link required C code to use OpenSSL.
  • Additional test tags and general test setup to allow the execution of appropriate tests with the new backend.

Additionally, the first set of algorithms, namely the message digests, are implemented. The aforementioned tags are, also, added to the corresponding tests.

Signed-off-by: Kostas Tsiounis kostas.tsiounis@ibm.com

@johnpeck-us-ibm

Copy link
Copy Markdown
Member

I know we talked about contexts and one for FIPS and one for non-FIPS. Could we use the Property query instead?
sha256 = EVP_MD_fetch(NULL, "SHA2-256", "fips=yes"); - FIPS
sha256 = EVP_MD_fetch(NULL, "SHA2-256", "provider=default"); - Non-FIPS.
Note the NULL for the context.

Does using this have any performance ramifications?

@KostasTsiounis
KostasTsiounis force-pushed the openssl_backend branch 2 times, most recently from 1b76bd7 to 31add61 Compare July 7, 2026 20:28
@KostasTsiounis
KostasTsiounis force-pushed the openssl_backend branch 2 times, most recently from 8920c23 to 0e09aab Compare July 21, 2026 15:39
@jasonkatonica
jasonkatonica requested a review from thu-ibm July 21, 2026 18:14
@KostasTsiounis
KostasTsiounis marked this pull request as ready for review July 31, 2026 00:05
Comment thread src/main/java/com/ibm/crypto/plus/provider/base/NativeImplementation.java Outdated
Comment thread src/main/java/com/ibm/crypto/plus/provider/ock/NativeOCKAdapterNonFIPS.java Outdated
Comment thread src/main/java/com/ibm/crypto/plus/provider/openssl/NativeOpenSSLAdapter.java Outdated
Comment thread src/main/native/openssl/Digest.c Outdated
Comment thread src/main/native/openssl/Digest.c
Comment thread src/main/java/com/ibm/crypto/plus/provider/openssl/NativeOpenSSLAdapter.java Outdated
Comment thread src/main/java/com/ibm/crypto/plus/provider/openssl/OpenSSLException.java Outdated
#include "Utils.h"
#include <string.h>

int debug = 0; // FIXME

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All gslogMessage calls always print to stderr regardless of the flag value

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is following the same pattern that all OCK native code has. Do you think we should change this to have different levels of logging? We could open an issue to do this everywhere if we want.

Comment thread src/test/java/ibm/jceplus/junit/tests/TestArguments.java
@KostasTsiounis
KostasTsiounis requested a review from taoliult August 6, 2026 20:46
@KostasTsiounis KostasTsiounis changed the title Openssl backend Support OpenSSL backend and implement message digest Aug 7, 2026
Comment thread src/main/native/openssl/Digest.c Outdated
* Signature: (JJI)V
*/
JNIEXPORT void JNICALL
Java_com_ibm_crypto_plus_provider_openssl_NativeOpenSSLImplementation_DIGEST_1digest_1and_1reset__JJI(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to the parameters, this should be _JJJI

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated comment and method signature.

static private ConcurrentLinkedQueueLong contexts[];

static private int runtimeContextNum[];
private static final Map<NativeInterface, Integer[]> runtimeContextNumPerBackend = new HashMap<>();

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a concern here by using two hashmaps. If I understand correctly, the backend cache state is published through two separate maps. Another thread can observe contextsPerBackend initialized before the corresponding runtimeContextNumPerBackend entry is published, skip initialization, and later dereference a null runtimeContextNum.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I follow the train of thought here. Are you talking about the static class variables? These will be assigned to a new hashmap during the class initialization. When a thread actually attempts to get a context, it will go through the double check and synchronization lock in lines 75-78. The context queue per backend is also a concurrent one.

Did you mean something else that I missed here?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I’m referring to the static per-backend maps, but the issue is not the initialization of the maps themselves or the ConcurrentLinkedQueue.

Consider two threads entering getContext() for a backend that has not been initialized yet. Both threads can initially read contexts == null and runtimeContextNum == null. Thread A acquires the lock first, initializes both entries, and releases the lock. Thread B then acquires the lock and re-reads only contextsPerBackend. Since contexts is now non-null, it skips the initialization block. However, Thread B never re-reads runtimeContextNumPerBackend, so its local runtimeContextNum variable is still null from the read before acquiring the lock.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe ConcurrentHashMap instead of Map ?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I make sure to set runtimeContextNum in the synchronized block if contexts has already been set.

goto cleanup;
}

rc = EVP_DigestFinal_ex(mdCtx, digestBytesNative, (unsigned int *)&digestLen);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any reason why we call EVP_DigestFinal_ex twice? If you want to get the output buffer size, i believe you can do something like

int digestLen = EVP_MD_CTX_get_size(mdCtx);

digestBytes = (*env)->NewByteArray(env, digestLen);

unsigned int actualLen = 0;

EVP_DigestFinal_ex(
    mdCtx,
    digestBytesNative,
    &actualLen);

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to suggested API. It looks like EVP_DigestFinal_ex doesn't guarantee proper size.

Comment thread src/main/native/openssl/Digest.c Outdated
Java_com_ibm_crypto_plus_provider_openssl_NativeOpenSSLImplementation_DIGEST_1update(
JNIEnv *env, jclass thisObj, jlong osslContextId, jlong digestId,
jbyteArray data, jint offset, jint dataLen) {
//static const char *functionName = "NativeOSSLImplementation.DIGEST_update";

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment can be removed.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed.

Comment thread src/main/java/com/ibm/crypto/plus/provider/openssl/NativeOpenSSLAdapter.java Outdated
Comment thread src/main/native/openssl/Digest.c Outdated
Comment on lines +299 to +302
if (digestLen < 0) {
throwOSSLException(env, 0, "DIGEST_digest_and_reset_internal: The specified data length is negative");
return 0;
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we can skip this check since a unsigned int could not be < 0

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed.

Comment thread src/main/native/openssl/Utils.c Outdated
void osslCheckStatus(void) {
unsigned long errCode;

while ((errCode = ERR_get_error()) == 1) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we sure all errors return as a value of 1 ? The OpenSSL doc for this API ERR_get_error indicates a value of 0 if there are no errors left to read from the queue.

while ((errCode = ERR_get_error()) != 0)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to suggestion.

Comment thread src/main/native/openssl/Utils.c Outdated
static char printBuffer[4096];

va_start(formatArgs, formatString);
charsPrinted = vsprintf(printBuffer, formatString, formatArgs);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we use vsnprintf instead of vsprintf in all our new openssl code? If formatString were large here or was missing a null terminator I think it would overlay printBuffer. In general all our new code we should try to make as bounded as possible. Something like the following in various places:

vsnprintf(printBuffer, sizeof(printBuffer), formatString, formatArgs);

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Switched to vsnprintf.

static private ConcurrentLinkedQueueLong contexts[];

static private int runtimeContextNum[];
private static final Map<NativeInterface, Integer[]> runtimeContextNumPerBackend = new HashMap<>();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe ConcurrentHashMap instead of Map ?

Comment thread src/main/native/openssl/Digest.c Outdated
Java_com_ibm_crypto_plus_provider_openssl_NativeOpenSSLImplementation_DIGEST_1digest_1and_1reset__JJI(
JNIEnv *env, jclass thisObj, jlong osslContextId, jlong digestId,
jlong digestBytes, jint length) {
//static const char *functionName = "NativeOSSLImplementation.DIGEST_digest_and_reset";

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason to keep these method names through the code? If we are going to use them then we should add debugging pathing, if not i think we could eliminate these. We could always later copy the value from the comment for the method for the Method: name such that the value would not be lost.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Followed the same pattern as OCK initially, but I removed them now.

@Override
public String getLibraryBuildDate() {
if (libraryBuildDate == unobtainedValue) {
libraryBuildDate = NativeOpenSSLImplementation.getLibraryBuildDate();;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two ;; characters.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

// value has not yet been obtained. We do this because some values
// may be null and we only want to query the value one time.
//
private static String libraryBuildDate = unobtainedValue;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this have to be static ? I could see how there are two threads trying to set libraryBuildDate below might cause some issues?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They would just set it to the same value. I don't think there is a need to keep one per instance.

EVP_MD_free(md);
md = NULL;

(*env)->ReleaseStringUTFChars(env, digestAlgo, digestAlgoChars);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might need a null check on digestAlgoChars here? If an exception is thrown prior to line 41 then digestAlgoChars would be null.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If an exception is thrown that early, we don't go to cleanup. We just return 0.

public static NativeOpenSSLAdapterNonFIPS getInstance() {
if (instance == null) {
instance = new NativeOpenSSLAdapterNonFIPS();
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it guaranteed that NativeOpenSSLAdapterNonFIPS.getInstance() returns a single instance under concurrent first access? getInstance() itself is not synchronized and its instance field is not volatile, so two threads may construct different adapter instances.

This seems particularly relevant now that Digest uses the NativeInterface instance itself as the key for contextsPerBackend. Two adapter instances representing the same OpenSSL backend would therefore create separate digest caches.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right. Changed to do a double check with synchronization in between.

With this change, all appropriate code is added to support
the use of OpenSSL as a backend. That includes:
- Main Java class to load the libraries required and offer
  native methods.
- Adapter Java classes to smoothly integrate without changes
  with existing code.
- Makefiles to compile and link required C code to use
  OpenSSL.
- Additional test tags and general test setup to allow the
  execution of appropriate tests with the new backend.

Additionally, the first set of algorithms, namely the
message digests, are implemented. The aforementioned
tags are, also, added to the corresponding tests.

Signed-off-by: Kostas Tsiounis <kostas.tsiounis@ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants