Provisioning cloud resources

Cloud providers offer lots of services to be used by developers. The mentioned services range from traditional virtual machines, through storage solutions to AI tooling. Each of them is just smaller or bigger building block which can be used when putting together more complex system.
When starting your journey with cloud you will probably be amazed with what you will see in various tutorials: with few clicks in AWS/Azure/Google cloud console you can provision all the stuff which is needed for running your cool application. In fact clicking through the cloud provider UI is great when learning and you just want to play with this stuff.
While great for learning purposes this approach does not scale when:
  • Working on bigger cloud project where you need to provision lots of stuff
  • Working with the team
  • You need to create additional environments and cloud infrastructure should be the same
The reason why the approach mentioned above does not scale is because it is manual process, and since its manual it is hardly reproducible. Imagine situation where you would like to setup additional testing environment which should be only accessible from internal network of your company. Even assuming that you have only few resources running this would be tedious job (creating storage, setting up networking rules, permissions etc.).

Fortunately for Azure we have Azure Resource Manager templates which help to address the issues described above. Note that there are also other tools available for performing this task e.g. Terraform - however these are out of scope for this article. 

What is Azure Resource Manager (ARM)?

Azure Resource Manager is a layer which sits between Azure resources (e.g. Blobs, Functions) and various interfaces such as:
  • Azure Portal
  • SDK's
  • REST API's
  • Command line tools
It is responsible for performing authentication and authorization of requests and forwarding them to services responsible for management of specific resources.

One of the benefits of ARM is that it allows you to manage you infrastructure with so called templates - these are JSON's containing definition of resources to be created/updated. This article will describe format and some basic features of ARM templates through simple example of creating Azure Storage Account with Blob container.

Login to Azure and setup default subscription

Lets start from logging in to your Azure account from Azure CLI. If you don't have tool installed, you will need to install it by following instruction from this page.
az login
The command will guide you in the process of logging in to your Azure account - after logging in you will be able to execute commands against your Azure account with Azure CLI. 

Next step will be setting default subscription - it is the subscription used by default when using the tool so that you don't have to provide it explicitly. In case you would like to use different subscription you will need to change default setting or explicitly provide different one.

az account list --output table
az account set --subscription "Subscription of your choice"
The first of the above commands will list all subscriptions you have access to formatted as table - which is quite useful when setting default subscription. Screenshot below presents sample output from the command.


The second command is simply setting default subscription, as you can see there is only single --subscription parameter - you can provide subscription name or id, which values can be found in the table generated by the previous command.

Create Resource Group

Before we jump into creation of the template we will create Resource Group which will contain resources we will provision later on. We will do that by using another command from Azure CLI.
az group create --location eastus --name my-infra-rg
The above command will instruct Azure to create my-infra-rg Resource Group in default subscription in East US region. Note that the location (region) you are setting when creating Resource Group determines where Azure is storing metadata (e.g. about deployments) for resources contained in the group - the resources itself can be located in any location you want. This has some implications - suppose that you have Resource Group in Central US and resources in East US - when the first goes down you will not be able to perform new deployments to that group, however the resources itself should be working properly.

After successfully finishing its job, the above command should provide output similar to one on the screenshot below.


The information which is important here are:

  • id - this is resource Id of the resource group you have just created. It includes subscription Id, which I obfuscated on the screenshot.
  • properties.provisioningState - this property informs you about the status of the provisioning. In this case it was successful.

Create ARM template

Now we are ready to start creating template itself. Create directory where you would like to store your template. In the directory create azuredeploy.json file and paste the content of the snippet below. You can use every code editor you want. In case you are using Visual Studio Code there is extension available for Azure Resource Manager which provides: syntax highlighting, code completion and tons of snippets helpful when creating ARM template.
  {
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {},
    "functions": [],
    "variables": {},
    "resources": [],
    "outputs": {}
}
There are several properties in created JSON file which configure different aspects of template:
  • parameters - contains values which allow you to re-use template e.g. across different environments
  • functions - ARM templates do contains lots of built-in helper functions which allow you to e.g. operate on text strings, extract keys and connection strings from other resources. Full list of available functions is available here. The property contains so called user-defined functions which are compositions of built in functions. 
  • variables - contains variables which you want to re-use inside your template. It is especially useful in case of values computed with functions.
  • resources - this is the most important property, it is where you define Azure resources to be created.
  • outputs - contains values you want to... well output after deployment. If the template is so called linked template (this is out of scope for this post, in short this is option for re-using templates), values listed in output section can be used in template which references linked templated. You can also use it when executing some automation.

Add Storage Account with Blob container

Now we will add definitions for Storage Account and Blob container associated with it. Add code from snippet below to resources section of azuredeploy.json file.
{
    "name": "provisionwarmtemplate",
    "type": "Microsoft.Storage/storageAccounts",
    "apiVersion": "2019-06-01",
    "tags": {
        "displayName": "provisioningWithArmTemplatesSA"
    },
    "location": "[resourceGroup().location]",
    "kind": "StorageV2",
    "sku": {
        "name": "Standard_LRS",
        "tier": "Standard"
    },
    "resources": [
        {
            "type": "blobServices/containers",
            "apiVersion": "2019-06-01",
            "name": "[concat('default/', 'my-storage-container')]",
            "dependsOn": [
                "[resourceId('Microsoft.Storage/storageAccounts', 'provisionwarmtemplate')]"
            ]
        }
    ]
}
There are few things to note about the code above:
  • It is creating two resources Storage Account with name provisionwarmtemplate and Blob container my-storage-container which is nested resource dependent on the Storage Account.
  • For Storage Account we are setting location property which will define region where it will be deployed. The location is set with use of function resourceGroup() which returns object describing resource group in which resource is being deployed. Luckily for us this object does contain location property. It is important to note that the function is only available when deploying at Resource Group level. If the value is being set with functions, the string is wrapped with square braces - when using Visual Studio Code extension for ARM, you will get syntax highlighting and completion for these.
  • Blob container my-storage-container is added as a nested resource to Storage Account - because of that it has to include dependsOn property. The property instructs Azure Resource Manager that it should create Storage Account first and then Blob container. This has to be explicitly set for nested resources. Similarly to location, the value is set with function - in this case it is resourceId function which resolves Id of the resources. First argument of the function is resource type, the second one resource name.
  • Both resources do include apiVersion property - it defines which version of the API should be used for handling deployment of specific resources. Each resource has its own distinct set of API versions. Newer versions of the API can have some additional features, on the other hand they may deprecate others.

Refactoring

If you look at the resulting code you will find that there is lots of hardcoded stuff here:
  • Name of the Storage Account
  • Name of the Blob container
  • Location of Storage Account - it is set to the same location as Resource Group where you are deploying
In one of the previous sections I have mentioned that ARM templates allow parametrization with parameters property. Lets leverage that feature to parametrize the template.

parameters property is an JSON object, the keys are parameter names and values are objects describing parameter. Lets take a closer look on that - the snippet below contains parameters property of the template filled with parameter for setting name of the Storage Account.
"parameters": {
    "storageAccountName": {
        "type": "string",
        "metadata": {
            "description": "Storage Account name"
        }
    }
}
The parameter above is described by two properties:
  • type - which defines type of the property value. You can fined complete list of types here.
  • metadata.description - description of the parameter - it is useful when you want to provide more information about how the parameter is being used.
Additionally you can also set list of allowed values (allowedValues property) or default value (defaultValue property).
When you define parameter you can reference it by using parameters() function e.g. "[parameters('storageAccount')]" - it accepts single argument which is name of the parameter. As an exercise, finish refactoring of the template. This GitHub repository contains code for this example.

The final step for the refactoring is creating parameters file which will be used when deploying resources. Create azuredeploy.parameters.json and paste snippet below into its content.

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "storageAccountName": {
            "value": "provisionwarmtemplate"
        },
        "location": {
            "value": "eastus"
        },
        "containerName": {
            "value": "my-storage-container"
        }
    }
}
With parameter files you can e.g. reuse template for performing deployment to different environments  (in such scenario you could have separate parameters file per environment).

Deployment

The last step is performing actual deployment. We will do that with Azure CLI. Command below will deploy resources defined in the template to my-infra-rg Resource Group using parameter values defined in azuredeploy.parameters.json.
az deployment group create --resource-group my-infra-rg --template-file .\azuredeploy.json --parameters .\azuredeploy.parameters.json
Now you should see newly created Storage Account in Azure Portal. Of course there is a way to list Storage Accounts from my-infra-rg Resource Group with Azure CLI, you can do that by running command below.
az storage account list --query "[?resourceGroup == 'my-infra-rg'].{Name:name, Location:location, ResourceGroup:resourceGroup}" --output table
When you are finished with playing with templates it is good idea to remove resources you will not be using. Since we have put all the resources for the exercise in my-infra-rg Resource Group, we can just delete the group - this operation will remove all the resources associated with it. For removing Resource Group we will use following command.
az group delete --name my-infra-rg

That's it for this article. Now you can go and play by creating template for e.g. project you are working on right now. There are tons of information about Azure Resource Manager in Microsoft docs (linked in More reading section below) and azure-quickstart-templates GitHub repository does contain some sample templates for provisioning common resources.

More reading