[Linux] Incompatibility with FIPS Enabled Systems Due to PKCS#1 v1.5 Padding

[Trautenberk]

Description

We are encountering an issue when running the Sustainsys.Saml2 package on a Linux host where FIPS is enabled. According to the recent FIPS-related document (NIST SP 800-131A), PKCS#1 v1.5 padding has been disabled. As a result, our application throws the following exception when we try to add certificate to ServiceCertificateCollection:

      using X509Certificate2 cert = LoadCertificate();
      var collection = new ServiceCertificateCollection();
      collection.Add(cert);

The exception:

Unhandled exception. Interop+Crypto+OpenSslCryptographicException: error:1C8000A5:Provider routines::illegal or unsupported padding mode
   at Interop.Crypto.RsaEncrypt(SafeEvpPKeyHandle pkey, ReadOnlySpan`1 source, RSAEncryptionPaddingMode paddingMode, IntPtr digestAlgorithm, Span`1 destination)
   at System.Security.Cryptography.RSAOpenSsl.TryEncrypt(SafeEvpPKeyHandle key, ReadOnlySpan`1 data, Span`1 destination, RSAEncryptionPadding padding, Int32& bytesWritten)
   at System.Security.Cryptography.RSAOpenSsl.Encrypt(Byte[] data, RSAEncryptionPadding padding)
   at Sustainsys.Saml2.Internal.CryptographyExtensions.Encrypt(XmlElement elementToEncrypt, Boolean useOaep, X509Certificate2 certificate)
   at Sustainsys.Saml2.Configuration.ServiceCertificateCollection.CertificateWorksForDecryption(X509Certificate2 certificate)
   at Sustainsys.Saml2.Configuration.ServiceCertificateCollection.InsertItem(Int32 index, ServiceCertificate item)
   at Sustainsys.Saml2.Configuration.ServiceCertificateCollection.Add(X509Certificate2 certificate)

I tried to track the source of the problem and found out a possible cause. In ServiceCertificateCollection.CertificateWorksForDecryption
it is hardcoded to use PKCS padding instead of OAEP:

When this argument is set to true, the package seems to work properly on Linux with FIPS enabled.

Non Security Issues

Please note that only critical compatibility fixes, such as when major browsers change behaviour, are fixed in 1.X or 2.X versions.

Information needed

1. What nuget packages are you using
2. What is the expected behaviour
3. What happens instead. In the case of an exception, this includes the exception typ, complete exception message (personal information may be redacted) and a stack trace.