Post-Quantum Cryptography in ADSS Server – HSM Deployment with Thales

How does Post‑Quantum Cryptography behave inside Hardware Security Module (HSM) environments—and how does that compare to software‑based key generation?

For high‑assurance PKI deployments, CA and signing keys are almost always backed by HSMs. Therefore, PQC adoption must be evaluated within secure hardware boundaries, not only in software.

This article covers:

• Raw ML‑DSA signature testing using Thales HSM
• Certificate and CRL signing via HSM‑backed keys
• Performance comparison (classical vs PQC inside HSM)
• Direct comparison of software vs hardware key generation

In the previous articles in this series, I examined:

• Structural integration of ML‑DSA into X.509 and CMS
• Interoperability validation with OpenSSL and Bouncy Castle
• Software‑based performance benchmarking

This final article addresses a critical production question. Let's get into it.

1. Why HSM Testing Matters

Software benchmarking provides insight into algorithmic cost. However, production trust services operate differently:

• Private keys are generated inside HSMs
• Key material is non‑exportable
• Signature operations occur across secure boundaries
• Compliance requirements often mandate hardware‑backed keys

In this context, PQC must be validated against three criteria:

• Can ML‑DSA keys be securely generated and stored inside the HSM?
• Can raw signature operations execute reliably through PKCS#11?
• What is the performance delta compared to RSA and ECDSA in hardware?

2. Test Architecture

The test environment consisted of:

• Sample Program (POC)
• Thales PCIe K7 HSM
• PKCS#11 interface (PKCS#11 v 3.2 DRAFT)
• ML‑DSA‑44, ML‑DSA‑65/87‑test keys
• RSA‑2048 and ECDSA‑P256 comparison keys

ADSS Server was configured to:

• Generate CA keys inside the HSM
• Perform certificate signing using HSM‑backed keys
• Sign CRLs via HSM
• Execute raw signature operations through PKCS#11

Software fallback crypto sources were not used during testing to ensure all private key operations occurred inside the secure boundary.

3. Raw ML‑DSA Signature Testing via HSM

3.1 Key Generation Inside the HSM

ML‑DSA‑44, ML‑DSA‑65/87‑key was generated directly within the HSM.

Validation confirmed:

• Private key material never left the hardware boundary
• Public keys were correctly exposed for certificate construction
• PKCS#11 handles were correctly mapped to ADSS key references

No key export operations were required.

3.2 Raw Signature Execution

Raw signing operations were performed using fixed payloads.

Results confirmed:

• ML‑DSA signature sizes matched expected parameter‑set output
• No truncation or encoding inconsistencies occurred
• Signatures verified successfully using OpenSSL 3.6.1 and Bouncy Castle 1.83

This demonstrates that ML‑DSA operations can be securely executed within an HSM boundary without structural or encoding deviations.

4. Certificate & CRL Signing via HSM

Beyond raw operations, full PKI workflows were validated.

4.1 CA Certificate Issuance

Using HSM‑backed ML‑DSA keys:

• Root/Issuing CA certificates were issued
• End‑entity certificates were signed

Validation confirmed:

• Correct ML‑DSA Algorithm Identifier encoding
• RFC 9881 compliance
• Successful path validation in OpenSSL

No structural differences were observed between software‑generated and HSM‑generated signatures.

4.2 CRL Signing

CRLs signed using HSM‑resident ML‑DSA keys were:

• Generated successfully
• Verified using OpenSSL
• Confirmed structurally compliant

CRL size increase was consistent with software‑based PQC results.

5. Performance Comparison – Classical vs PQC in HSM

Performance was measured for:

• RSA‑2048 signing (HSM)
• ECDSA‑P256 signing (HSM)
• ML‑DSA‑44 (Level 2) signing (HSM)
• ML‑DSA‑65 (Level 3) signing (HSM)
• ML‑DSA‑87 (Level 5) signing (HSM)

5.1 Raw Signing Performance: HSM versus Software

Observations

• ECDSA performance is higher than ML‑DSA, as expected.
• Performance scales predictably with security level.
• For low‑frequency CA signing operations, impact is negligible.
• For high‑volume issuing CAs, throughput planning becomes important.

The behaviour inside HSM closely mirrors relative performance trends observed in software, with additional boundary overhead.

6. Software vs Hardware Key Generation – Direct Comparison

Post‑Quantum adoption introduces an architectural decision: Should ML‑DSA and ML‑KEM keys be generated in software or inside the HSM?

Both approaches are technically supported by ADSS Server, but they differ significantly in assurance and operational guarantees.

6.1 Security Boundary

Software‑Based Key Generation

When keys are generated in software:

• Private key material exists in application memory
• Entropy is sourced from the operating system CSPRNG
• Key protection depends on OS hardening and access controls
• Keys may later be imported into an HSM (if permitted)

Risk exposure includes:

• Memory scraping attacks
• Insider access risks
• Host‑level compromise

Software generation is appropriate for:

• Test environments
• Performance benchmarking
• Ephemeral keys
• Lower‑assurance internal systems

HSM‑Based Key Generation

When keys are generated inside a Thales HSM:

• Private key material never leaves the secure boundary
• Entropy is generated by hardware‑based TRNG
• Non‑exportability can be enforced
• Tamper‑resistant protections apply

For regulated PKI environments (e.g., QTSP, eIDAS, Root CAs): HSM‑based key generation is typically mandatory.

6.2 Entropy Considerations for PQC

ML‑DSA and ML‑KEM require:

• Larger random inputs
• Strong uniformity guarantees
• High‑quality entropy sources

In software:

• Entropy relies on OS DRBG quality

In HSM:

• Entropy is hardware‑generated
• Often FIPS‑certified

For long‑lived CA keys, entropy quality is a critical design factor.

6.3 Key Generation Performance

Relative behaviour observed:

Observations:

• HSM key generation is slower due to secure boundary enforcement.
• PQC key generation is heavier than ECDSA in both environments.
• Key generation frequency for CA keys is low, making performance impact negligible in most production scenarios.

Security posture, not speed, should drive the decision.

7. Architectural Implications

From an operational standpoint:

Key Size

ML‑DSA keys are larger than ECDSA keys, affecting:

• HSM storage capacity
• Backup duration
• Replication processes

Throughput

PQC reduces signing throughput per HSM partition. Mitigations include:

• Horizontal scaling
• Partition allocation strategies
• Load‑balanced ADSS signing services

Trust Model

The PKIX trust model remains unchanged. Only the cryptographic primitive differs.

Conclusion

Post‑Quantum Cryptography must be validated not only in software but within secure hardware environments to be considered production ready.

Testing with Thales HSM infrastructure confirms that:

• ML‑DSA keys can be securely generated inside hardware
• Raw signature operations function correctly via PKCS#11
• Certificate and CRL signing workflows remain standards‑compliant
• Performance remains operationally manageable
• Software vs HSM key generation presents a clear assurance trade‑off

From an architectural perspective:

The transition to PQC in HSM‑backed PKI systems is primarily a capacity and assurance planning exercise — not a redesign of the trust infrastructure.

Sample HSM Program (POC) but soon will be available in ADSS Server demonstrates that quantum‑safe signing can be deployed across both software and hardware‑backed trust environments today.