Skip PQC connection tests when the runtime cannot negotiate PQC#9643
Conversation
OpenSSL >= 3.5 can ship ML-KEM and ML-DSA while still keeping them out of the default negotiation lists, for example under RHEL's system-wide crypto policies. The tests force a PQC-only server but connect with the default gem fetcher, so they fail the handshake on such hosts even though the version gate passes. Probe a real loopback handshake instead. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Thank you for fixing this issue. Sorry, I missed the case that OpenSSL disables ML-DSA for TLS by default on OpenSSL >= 3.5 by the OpenSSL configuration. We discussed the case on ruby/openssl#964 before. |
|
Oh, I also forgot that report. |
|
Perhaps not only ML-DSA (PQC signature) but also ML-KEM (key exchange) may be disabled in OpenSSL >= 3.5 on RHEL 9. I will investigate. |
@hsbt (CC: @rhenium ), I investigated above thing on a RHEL 9.8 server in my company. Difference of OpenSSL config files between security policy DEFAULT (PQC disabled) and DEFAULT:PQ (PQC enabed)First, in the Fedora/RHEL OpenSSL RPM, the used OpenSSL config file is Below is the case of Ruby RPM ( Below is the case of upstream Ruby compiled with upstream OpenSSL. Then the /etc/pki/tls/openssl.cnf You can check the current crypoto policy by the following command. In RHEL 9.x >= 9.7, the security policy DEFAULT disables PQC (both PQC signatures: ML-DSA and PQC key exchange: ML-KEM). The The content is below. Then I ran the following command to change the security policy from DEFAULT to DEFAULT:PQ. The DEFAULT:PQ is a sub security policy to enable PQC in RHEL 9.x >= 9.7. This is not the case of RHEL-10. RHEL-10 enables PQC in security policy DEFAULT. I usually don't recommend to run the following command in RubyCI's RHEL 9 server, because it recommends restarting OS after that, and maybe we want to test in the default RHEL 9 server Now the The content is below. The difference of the $ diff -u opensslcnf.txt.default opensslcnf.txt.default_pq
--- opensslcnf.txt.default 2026-03-04 10:13:14.000000000 +0000
+++ opensslcnf.txt.default_pq 2026-06-26 15:50:55.000000000 +0100
@@ -4,5 +4,5 @@
TLS.MaxProtocol = TLSv1.3
DTLS.MinProtocol = DTLSv1.2
DTLS.MaxProtocol = DTLSv1.2
-SignatureAlgorithms = ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:ed25519:ed448:rsa_pss_pss_sha256:rsa_pss_pss_sha384:rsa_pss_pss_sha512:rsa_pss_rsae_sha256:rsa_pss_rsae_sha384:rsa_pss_rsae_sha512:RSA+SHA256:RSA+SHA384:RSA+SHA512:ECDSA+SHA224:RSA+SHA224
-Groups = *X25519:secp256r1:X448:secp521r1:secp384r1:ffdhe2048:ffdhe3072:ffdhe4096:ffdhe6144:ffdhe8192
+SignatureAlgorithms = ?mldsa44:?mldsa65:?mldsa87:ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:ed25519:ed448:rsa_pss_pss_sha256:rsa_pss_pss_sha384:rsa_pss_pss_sha512:rsa_pss_rsae_sha256:rsa_pss_rsae_sha384:rsa_pss_rsae_sha512:RSA+SHA256:RSA+SHA384:RSA+SHA512:ECDSA+SHA224:RSA+SHA224
+Groups = *?X25519MLKEM768:?x25519_mlkem768:?SecP256r1MLKEM768:?p256_mlkem768:?SecP384r1MLKEM1024:?p384_mlkem1024/*X25519:secp256r1:X448:secp521r1:secp384r1:ffdhe2048:ffdhe3072:ffdhe4096:ffdhe6144:ffdhe8192In signature algorithms, ML-DSA-NN (mldsa44, mldsa65 and mldsa87) are added. In groups, ML-KEM groups (X25519MLKEM768, x25519_mlkem768, SecP256r1MLKEM768, p256_mlkem768, SecP384r1MLKEM1024, p384_mlkem1024) are added. A logic to skip the case that PQC is disabled in OpenSSL config file (the cases of RHEL 9.x >= 9.7)I think the current skipping logic to trying to handshake is the best, while the logic may have a challenge that we cannot judge the handshake fails due to disabled PQC in the OpenSSL config or a bug related to PQC, and the logic is also costy when other ruby/* libraries implement this logic. I created the following proof-of-concept script to check if the check_pqc.rb require 'openssl'
config = OpenSSL::Config.load(OpenSSL::Config::DEFAULT_CONFIG_FILE)
pqc_supported = false
if config.sections.include?('crypto_policy')
if config.get_value('crypto_policy', 'SignatureAlgorithms')&.include?('mldsa65') \
&& config.get_value('crypto_policy', 'Groups')&.include?('X25519MLKEM768')
pqc_supported = true
end
else
pqc_supported = true
end
puts "pqc_supported: #{pqc_supported}"The script prints false in RHEL 9 DEFAULT security policy: The script prints true in RHEL 9 DEFAULT:PQ security policy: With upstream OpenSSL where there are But the problem is this script depends on Fedora/RHEL specific config structure with crypto_policy section. It's not good to apply this logic to the ruby project. |
|
I opened the issue ticket ruby/openssl#1075. |
2 participants
The PQC connection tests added in #9615 fail on RHEL9 chkbuild with an SSL handshake failure even though the OpenSSL version gate passes. RHEL9 ships OpenSSL 3.5.5, and OpenSSL's own ML-KEM and ML-DSA tests pass there, so the algorithms themselves are available. The problem is that the tests force a PQC-only server (X25519MLKEM768 plus an ML-DSA-65 certificate) while connecting with the default gem fetcher. OpenSSL 3.5 can ship those algorithms while still keeping them out of its default negotiation lists, for example under RHEL's system-wide crypto policies, so a default client cannot agree with the forced server and the handshake fails.
A version number check cannot detect a policy-disabled algorithm, so omit_unless_support_pqc now probes a real loopback handshake between a forced-PQC server and a default client. The PQC tests run only when that probe succeeds. This keys off actual capability rather than OS or version, so it also starts running automatically once a distribution enables PQC in its defaults.
http://rubyci.s3.amazonaws.com/rhel9/ruby-master/log/20260624T003003Z.fail.html.gz