util: update AdvancedTlsX509KeyManager to support key alias for reloaded cert#12686
Conversation
a54cf0b to
a18d1a6
Compare
a18d1a6 to
9456de4
Compare
|
I don't think using AdvancedTlsX509KeyManager with OpenSslCachingKeyMaterialProvider is all that good an approach with this, because it essentially leaks all prior keys in memory. If the life of the process after a year has 5 versions, that's not that big of a deal. But if it changes ever hour, that's not good. I don't think we're going to put random detection code in AdvancedTlsX509KeyManager to just warn you that things have gone wrong either. |
Yes, I think it should be used with caution. As for AdvancedTlsX509KeyManager itself, I think we still need to handle alias changes? |
Yes, that's tracked by #12485 . This PR doesn't actually address that, as it ignores the alias being used. I imagined keeping one older version around to handle in-progress handshakes. Since you're talking about every configuration having its own alias name, if the key manager mutates faster than handshakes complete (such that the alias is really old) then we could trigger an error in some way.
Okay. I'm fine with using a new alias each mutation in this key manager, but that should be consistent; for example, getPrivateKey() shouldn't return the new key when passed the old alias. That applies equally to all methods that receive the alias. And we won't have the "this sure seems like a lot of mutations" warning. At that point, if someone so desires they can improve OpenSslCachingKeyMaterialProvider to become an LRU cache (or introduce a new class). Then you could use a cache size of two. |
I think the idea of "keeping one older version around" makes sense. Also removed the warning part. |
ejona86
left a comment
There was a problem hiding this comment.
Looks good. Only minor comments.
|
Thank you! |
…ded cert (grpc#12686) ## Overview Make the alias in `AdvancedTlsX509KeyManager` dynamic so it can be used with Netty's `OpenSslCachingX509KeyManagerFactory` to update key material after reload. Fixes grpc#12670 Fixes grpc#12485 ## Problem When using `SslProvider.OPENSSL`, each TLS handshake must encode Java key material into a native buffer consumed by OpenSSL, which can account for ~8% of server CPU. Netty's `OpenSslCachingX509KeyManagerFactory` avoids this by caching the encoded buffer keyed by alias — but the previous implementation always returned `"default"`, so the factory could never detect credential rotations and create a new cache entry on cert reload. ## Details - The alias is now set to `key-<N>` (e.g. `key-1`, `key-2`, ...) and incremented on every `updateIdentityCredentials` call, ensuring the same alias always maps to the same key material. - One prior key value is kept to allow consistent handshaking during key changes.
… cert rotation (#16523) ### Motivation: The current `OpenSslCachingKeyMaterialProvider` does not evict stale entries after a cert rotation. This is related to a performance concern when using grpc-java (grpc/grpc-java#12670) ### Modification: Added `evictStaleEntries()`, which removes cached entries whose alias is no longer recognized by the `X509KeyManager`. It is called on a cache miss when new material is successfully loaded, so stale entries from rotated credentials are pruned before inserting the new one. ### Result: Better support for cert rotation. Related discussion: grpc/grpc-java#12686 grpc/grpc-java#12670 --------- Co-authored-by: Chris Vest <[email protected]>
…e entries after cert rotation (#16802) Auto-port of #16523 to 5.0 Cherry-picked commit: 40b824b --- ### Motivation: The current `OpenSslCachingKeyMaterialProvider` does not evict stale entries after a cert rotation. This is related to a performance concern when using grpc-java (grpc/grpc-java#12670) ### Modification: Added `evictStaleEntries()`, which removes cached entries whose alias is no longer recognized by the `X509KeyManager`. It is called on a cache miss when new material is successfully loaded, so stale entries from rotated credentials are pruned before inserting the new one. ### Result: Better support for cert rotation. Related discussion: grpc/grpc-java#12686 grpc/grpc-java#12670 Co-authored-by: wzhang <[email protected]> Co-authored-by: Chris Vest <[email protected]>
Overview
Make the alias in
AdvancedTlsX509KeyManagerdynamic so it can be used with Netty'sOpenSslCachingX509KeyManagerFactoryto update key material after reload.Fixes #12670
Problem
When using
SslProvider.OPENSSL, each TLS handshake must encode Java key material into a nativebuffer consumed by OpenSSL, which can account for ~8% of server CPU. Netty's
OpenSslCachingX509KeyManagerFactoryavoids this by caching the encoded buffer keyed by alias —but the previous implementation always returned
"default", so the factory could never detectcredential rotations and create a new cache entry on cert reload.
Details
key-<N>(e.g.key-1,key-2, ...) and incremented on everyupdateIdentityCredentialscall, ensuring the same alias always maps to the same key material.AdvancedTlsX509KeyManager(int revisionWarningThreshold)allows customizingthe soft warning threshold (default: 1024, matching
OpenSslCachingX509KeyManagerFactory'sdefault
maxCachedEntries). A warning is logged when the counter reaches the threshold, sincebeyond that point new aliases won't be cached and per-handshake encoding overhead resumes. The
key manager remains fully functional past this threshold.
nullbefore any credentials are loaded.