It's so easy - backup build and release definitions from VSTS using Azure Functions

When working with build and release definitions in VSTS we're blessed with the possibility to check audit logs, what was changed, when and by who. This - together with proper permissions setup - allow proper access management and easy rollback if something was misused. Unfortunately VSTS lacks an easy way to export those definitions so we can backup them or version in our repository. In this post I'll show you a quick way to schedule daily backups using Azure Functions.

Prerequisities

To perform actions from this post you'll need Visual Studio 2017 15.3 with Azure Functions SDK installed. Since those tools are no longer in preview, I no longer use CSX to create examples and proofs of concepts. I strongly advise you to update to the latest VS version so you can take the most from the new SDK.

What is more you'll need also a personal access token(PAT) from VSTS. Please read this article if you haven't for an idea how to get it.

Creating functions

To be able to schedule our backup, we'll need two functions. Both we'll be triggered by a timer and both will upload a blob to a Blob Storage container. Here's our infrastructure needed:

ARM template visualization created by ARMata

As you can see this is the basic infrastructure needed to be able to use Functions, which can be easily set up in Azure Portal. Once we have required components provisioned, we can prepare code, which will create backups.

In VS when you go to Create project wizard, you'll see a window with available templates. When you go to the Cloud tab you should see Azure Functions template ready to be created:

Once a project is created right-click on it, to to Add menu and select New item:

From the available positions select Azure Function and click Add. You'll see plenty of different function templates, from which we have to choose Timer trigger. Change the schedule to 0 0 0 */1 * * so it will be triggered once a day and click Ok.

Creating a backup

To create a backup we'll use once more VSTS REST API. Here are endpoint, which we'll use here:

They return JSON definitions, which can be easily stored and versioned. The actual code for creating a build definition backup looks like this:

/
public static class BuildBackup
{
	private const string Personalaccesstoken = "PAT";

	[FunctionName("BackupBuild")]
	public static async Task Run([TimerTrigger("0 */1 * * * *")]TimerInfo myTimer, [Blob("devops/build.json", FileAccess.Write)] Stream output, TraceWriter log)
	{
		try
		{
			using (var client = new HttpClient())
			{
				client.DefaultRequestHeaders.Accept.Add(
					new MediaTypeWithQualityHeaderValue("application/json"));

				client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic",
					Convert.ToBase64String(
						System.Text.Encoding.ASCII.GetBytes(
							string.Format("{0}:{1}", "", Personalaccesstoken))));

				using (var response = await client.GetAsync(
					$"https://{instance}.visualstudio.com/DefaultCollection/{project}/_apis/build/definitions?api-version=2.0")
				)
				{
					var data = await response.Content.ReadAsAsync<JObject>();
					foreach (var pr in data.SelectToken("$.value"))
					{
						var id = pr.First.SelectToken("$.id");
						using (var release = await client.GetAsync(
							$"https://{instance}.visualstudio.com/DefaultCollection/{project}/_apis/build/definitions/{id}?api-version=2.0")
						)
						{
							release.EnsureSuccessStatusCode();
							var releaseData = await release.Content.ReadAsStringAsync();
							var bytes = Encoding.UTF8.GetBytes(releaseData);
							await output.WriteAsync(bytes, 0, bytes.Length);
						}
					}
				}
			}
		}
		catch (Exception ex)
		{
			log.Info(ex.ToString());
		}
	}
}

To create a backup of a release definition you can use following function:

/
public static class ReleaseBackup
{
	private const string Personalaccesstoken = "PAT";

	[FunctionName("BackupRelease")]
	public static async Task Run([TimerTrigger("0 0 0 */1 * *")]TimerInfo myTimer, [Blob("devops/release.json", FileAccess.Write)] Stream output, TraceWriter log)
	{
		try
		{
			using (var client = new HttpClient())
			{ 
				client.DefaultRequestHeaders.Accept.Add(
					new MediaTypeWithQualityHeaderValue("application/json"));

				client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic",
					Convert.ToBase64String(
						Encoding.ASCII.GetBytes(
							string.Format("{0}:{1}", "", Personalaccesstoken))));

				using (var response = await client.GetAsync(
					"https://{instance}.vsrm.visualstudio.com/{project}/_apis/Release/definitions")
				)
				{
					var data = await response.Content.ReadAsAsync<JObject>();
					foreach (var pr in data.SelectToken("$.value"))
					{
						var id = pr.First.SelectToken("$.id");
						using (var release = await client.GetAsync(
							$"https://{instance}.vsrm.visualstudio.com/{project}/_apis/Release/definitions/{id}")
						)
						{
							release.EnsureSuccessStatusCode();
							var releaseData = await release.Content.ReadAsStringAsync();
							var bytes = Encoding.UTF8.GetBytes(releaseData);
							await output.WriteAsync(bytes, 0, bytes.Length);
						}
					}
				}
			}
		}
		catch (Exception ex)
		{
			log.Info(ex.ToString());
		}
	}
}

Some details:

  • I used a blob container named devops  - of course you can use any name you like
  • Unfortunately there's no way to combine those two functions(as long as you'd like to use different blob for holding build and release definitions)
  • You can easily version those JSON definitions by - instead of storing them in Blob Storage - calling a VSTS REST API for a repository and uploading a blob there

New Azure Functions SDK and VSTS - how to build your functions?

With recent release of Visual Studio 2017 15.3 we're given the possibility to use Azure Functions SDK, which is now fully integrated with IDE. This improves development greatly as we no longer have to maintain function.json by ourselves and are able to run function locally using in-built runtime. How about integrating new features into existing CI/CD pipeline? Well, there're some gotchas, fortunately you can easily configure things so everything runs smoothly.

Building a function project

When you create a new function project in VS you're given the options to easily add new functions with a boilerplate code. If you investigate .csproj file(last time I wrote about building Azure Functions using VSTS it was a .funproj file), you'll realize, that it differs greatly compared to legacy .csproj files:

/
<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net461</TargetFramework>
  </PropertyGroup>
  <ItemGroup>    
    <PackageReference Include="Microsoft.NET.Sdk.Functions" Version="1.0.0" />
  </ItemGroup>
  <ItemGroup>
    <Reference Include="Microsoft.CSharp" />
  </ItemGroup>
  <ItemGroup>
    <None Update="host.json">
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
    </None>
    <None Update="local.settings.json">
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
    </None>
  </ItemGroup>
</Project>

Considering new structure you can imagine, that it's going to give us some troubles. If you try to build it in VSTS, you'll get following error:

/
Error MSB4066: The attribute "Version" in element <PackageReference> is unrecognized.

Apparently using a default agent queue won't work here - agents don't have proper targets, which are required to build a project(if you build a project locally, you can find those targets in C:\Users\{USER}\.nuget\packages\microsoft.net.sdk.functions\1.0.0\build\netstandard1.0\Microsoft.NET.Sdk.Functions.Build.targets). It'd possible to just import them and run on a default agent, still this looks like a workaround. However, there's an easier solution - during scheduling of your build you can choose a different queue:

Let's try to build our project once more...

/
 Error MSB4041: The default XML namespace of the project must be the MSBuild XML namespace. If the project is authored in the MSBuild 2003 format, please add xmlns="http://schemas.microsoft.com/developer/msbuild/2003" to the <Project> element. If the project has been authored in the old 1.0 or 1.2 format, please convert it to MSBuild 2003 format.

There's still a problem - MSBuild is not able to determine what kind of XML is this. To fix this you can just modify <Project> node to following:

/
<Project Sdk="Microsoft.NET.Sdk" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

Now your project should build correctly:

When working on this solution I found, that some people tried to build new Azure Functions projects using dotnet build step in VSTS - unfortunately this didn't work for me. Let me know if you found another way of integrating new version of .csproj with VSTS!