Note: Slides do not tell the whole story of the talk, so take the stand alone slides with a grain of salt. Things may be taken out of context.
Month: May 2022
Improving Azure Key Vault Performance in ASP.NET Core By Up To 10x
If you’re using the
Azure.Extensions.AspNetCore.Configuration.Secrets package to plugin Azure Key Vault into
IConfiguration, this integration uses a number of different authenticaton options to authenticate to Azure Key Vault. By excluding the ones you aren’t using, you can drastically improve your startup time from ~15 seconds to ~1.5 seconds in my experience, depending on the ones you exclude.
I have listed all the options below, but commented them out, because which ones you exclude will depend on your situation.
What is Azure Key Vault?
Azure Key Vault is a secret management store hosted in Azure for storing sensitive information such as secrets that are needed for your application to function. These secrets can be things such as database connection strings, API keys, passwords, and other sensitive information that you do not want falling into the wrong hands. Azure Key Vault solves numerous problems with secret management – including secure storage (i.e. not in source control), access control, audit logging, versioning, and more.
Azure Key Vault can store things like secrets, certificates, and keys. In this blog post, we’re going to focus on secrets.
Azure Key Vault + ASP.NET Core = ♥
Azure Key Vault offers a tight integration with ASP.NET Core by way of a Configuration Provider that plugs into the
IConfiguration system in ASP.NET Core. You can find more details here, but in a nut shell the
Azure.Extensions.AspNetCore.Configuration.Secrets package offers a single method called
AddAzureKeyVault that allows you to specify your key vault name and how to authenticate.
This allows the secrets to all be loaded a single time for the lifetime of your application.
Loading secrets one time has numerous benefits:
- Keeping costs low. At the time of this writing you pay $0.03 USD per 10,000 retrievals, so paying that cost once per secret per application start would be much less than once per secret per request for any application with reasonable traffic.
- Performance on a per request basis is improved. You are not round-tripping to Azure Key Vault on each request. Instead you are paying that roundtrip cost once at application start. Of course, this comes with a tradeoff that if you update your secrets, you will have to restart your application. If this limitation is hard for your application to accept, you can set a ReloadInterval property to reload secrets on a specified
Below I’m connecting to a Azure Key Vault using the new
DefaultAzureIdentity which is a managed identity that works for Azure resources (such as Azure App Service) as well as local development, provided you’re logged into the Azure CLI via
This will result in all of your secrets being available via
IOptions in your ASP.NET Core application. The name of the secret in Azure Key Vault will match the name of the secret exposed in
IConfiguration. If you have nested values (i.e.
Parent:Child), those will be represented in Azure Key Vault with double dashes such as
The Problem – by default it can take 15 seconds or longer
The problem is, this can take ~15 seconds or longer in my experience (keep in mind, your performance may vary depending on a number of factors – including the number of secrets in your Azure Key Vault, your geographic location relative to the Azure Key Vault, etc.). Not only does this affect your application’s startup, but it can also slow down your tests if you’re using Azure Key Vault in combination with things like
The Solution – Exclude Authentication Credential You’re Not Using
DefaultAzureCredential will try many different authentication credentials to try and work out which one to use. You can view the source code here, but each one has different impacts on the performance of your application.
The order of authentication credentials it attempts is as follows:
- Environment variables
- Managed Identity
- Shared Token
- Visual Studio
- Visual Studio Code
- Azure CLI
- Azure Powershell
- Interactive Browser
You will have to figure out which make sense to exclude for your scenarios. For my scenario, I authenticate locally with the CLI via
az login, but when deployed to an Azure App Service, I leverage the Managed Identity (which is essentially the user your application is running as). That means I can ignore the rest as implemented in the code below:
Before making this change, most of the time I was getting ~15 seconds for an Azure Key Vault with 10 secrets. After making this change it dropped it to ~6 seconds.
Optimizing even further by detecting the ASP.NET Core Environment
However, I really only use the Azure CLI locally and the Managed Identity out in Azure App Service. On most applications, I usually create a custom environment called “Local” for my local development, which means I can use that to tell Azure KeyVault to use the CLI locally and Managed Identity anywhere else:
This drops the performance of my local development to just 1.5 seconds. This is a savings of ~13.5 seconds of what I was consistently seeing.
Don’t Forget To Benchmark Your Scenario
Again, your mileage may vary depending on which options you exclude, how many secrets you have, your geographic location relative to the Azure Key Vault, and lots of other factors.
As with anything related to performance, make sure to benchmark before and after to ensure that you’re gaining performance, because your exact scenario may behave differently than my scenarios.
Why not user dotnet user-secrets for local development?
I hear this question a lot – why not use
dotnet user-secrets for local development? I think that
dotnet user-secrets is not very team friendly in an active codebase. Here’s why:
- Any time anyone adds a new secret or changes a value for an existing one, they have to let the whole team know the key and value to add to the local user secret store
- Any time we onboard someone, we have to usually copy the user secrets from one machine to another
Using an Azure Key Vault specific for local development, both problems are solved. Anytime someone adds a new secret or changes a value to an existing one to Azure Key Vault, the whole team gets it. Also onboarding is as simple as adding someone to an Azure Active Directory Group that has access to that Azure Key Vault and you’re done.
Hope this helps!