Azure Key Vault - making it right

Recently I spent a couple of hours struggling with one of our Azure Key Vaults - its access policies to be more specific. Short story - I was unable to create a new one from the portal, nor access secrets or keys. After exchanging several emails with Microsoft and few calls with their technician, we managed to find both a quick solution and the root of all evil - invalid ARM template.

Key Vaults are somehow fragile resources when it comes to determining who can access them. You can be a "superadmin hero", still if a Key Vault was created in the other tenant or for an object not related with you, you can see it, you can manage it but if it comes to retrieving keys or secrets, you won't be able to do so. It's perfectly fine - it should be as secure as possible - but the way it informs you, that something is wrong, can be described as... well, lacking.

Consider following example:

{
	"type": "Microsoft.KeyVault/vaults",
	"name": "[parameters('keyVaultName')]",
	"location": "[resourceGroup().location]",
	"apiVersion": "2015-06-01",
	"tags": {
		"displayName": "my-keyvault"
	},
	"properties": {
		"enabledForDeployment": true,
		"enabledForDiskEncryption": true,
		"enabledForTemplateDeployment": true,
		"tenantId": "[parameters('tenantId')]",
		"accessPolicies": [
			{
				"tenantId": "[parameters('tenantId')]",
				"objectId": "[parameters('objectId')]",
				"permissions": {
					"keys": "[parameters('keysPermissions')]",
					"secrets": "[parameters('secretsPermissions')]"
				}
			}
		],
		"sku": {
			"name": "[parameters('vaultSkuName')]",
			"family": "A"
		}
	}
}

It is possible with ARM to create a key vault, which can be accessed e.g. by a group of admins(by passing correct tenantId and objectId). However, let's say you have made a mistake in those fields. Key vault will still be created with an access policy, but following things will happen also:

  • you won't be able to create a new access policy
  • you won't be able to browse keys
  • you won't be able to browse secrets
  • ARM will still be able to retrieve keys and secrets

Trying to add a new policy will result in "Invalid parameter name 'accessPolicy'" error while managing it with Azure Powershell CLI will give "401 Unauthorized" response. I've seen more descriptive messages in my life TBH.

Microsoft guided me to the solution with their article Change a key vault tenant ID after a subscription move, which describes a solution suitable also if a subscription hasn't been moved and a key vault has just been created(for a invalid tenant or an object). Performing steps from the article helped me in restoring proper access to the key vault and finally led to the proper solution of our issue.

Monitoring available threads for an application in Azure

Recently I've been thinking how can I track amount of threads available for an application hosted in Azure cloud. It was due to early discovered performance issues, which should be addressed sooner than later. Since we've decided to be async-heavy and made many possible breaking changes, solid metrics related to different application domains should be introduced and implemented.

Since we're taking advantage of Application Insights when measuring performance, finding a way to utilize it and measure threads available for our web app seems natural and uncomplicated. Fortunately Microsoft.ApplicationInsights NuGet package contains ITelemetryInitializer, which looks like this:

using Microsoft.ApplicationInsights.Channel;

namespace Microsoft.ApplicationInsights.Extensibility
{
  /// <summary>
  /// Represents an object that initializes <see cref="T:Microsoft.ApplicationInsights.Channel.ITelemetry" /> objects.
  /// </summary>
  /// <remarks>
  /// The <see cref="T:Microsoft.ApplicationInsights.DataContracts.TelemetryContext" /> instances use <see cref="T:Microsoft.ApplicationInsights.Extensibility.ITelemetryInitializer" /> objects to
  /// automatically initialize properties of the <see cref="T:Microsoft.ApplicationInsights.Channel.ITelemetry" /> objects.
  /// </remarks>
  public interface ITelemetryInitializer
  {
    /// <summary>
    /// Initializes properties of the specified <see cref="T:Microsoft.ApplicationInsights.Channel.ITelemetry" /> object.
    /// </summary>
    void Initialize(ITelemetry telemetry);
  }
}

All it needs is to implement Initialize() method.

Worth noting is the fact, that such initializer will be hit by all traces so don't go crazy when using it to log data from external resources etc.

With such interface, creating a custom initializer, which will log information about threads is a piece of cake:

//
public class AvailableThreadsInitializer : ITelemetryInitializer
{
	public void Initialize(ITelemetry telemetry)
	{
		int workerThreadsMax;
		int completionPortThreadsMax;
		int workerThreadsAvailable;
		int completionPortThreadsMaxAvailable;

		ThreadPool.GetMaxThreads(out workerThreadsMax, out completionPortThreadsMax);
		ThreadPool.GetAvailableThreads(out workerThreadsAvailable, out completionPortThreadsMaxAvailable);

		telemetry.Context.Properties["Max. threads"] = workerThreadsMax.ToString();
		telemetry.Context.Properties["Max. threads(async I/O)"] = completionPortThreadsMax.ToString();
		telemetry.Context.Properties["Available threads"] = workerThreadsAvailable.ToString();
		telemetry.Context.Properties["Available threads(async I/O)"] = completionPortThreadsMaxAvailable.ToString();
	}
}

All what is left to do is to add this initializer to the collection of already defined initializers:

//
TelemetryConfiguration.Active.TelemetryInitializers.Add(new AvailableThreadsInitializer());

Thanks to this change, each trace will have an information about threads available for your application - for sure it will ease diagnosis when something is wrong with your web application.

Thanks to @marekgrabarz for revealing this feature to me!