In this article, let us examine how to use Azure Resource Manager(ARM). Microsoft Azure came up with the latest service that deploys & manages the development work through the code. We might have heard of IaaS, SaaS, PaaS. Similarly, Microsoft Azure introduced IaaC(Infrastructure as a Code), which allows you to describe your infrastructure in a declarative way, store it in version control and modify it in a controlled manner.
The process minimizes the manual errors, increases development efficiency in deploying and maintaining the resources, By deploying a template file, one can easily deploy complete data centers within a few minutes rather than spending a lot of time through the portal. Let us understand the working of the ARM with the Data factory.
- An ARM template is very easy to understand and work with as it uses declarative language like JSON.
- Azure bicep file(written in bicep language) is much easier to understand and work when compared to JSON. But it is in the experimental stage developed by Microsoft.
Key Terminologies
- Azure Resource: It is an entity(like VMs, SQL Dbs, Virtual Networks and Containers, etc) managed by Microsoft Azure.
- Azure Resource Group: A resource group will have ‘n’ number of resources under it. And each and every resource should belong to any of the resource groups.
- Azure Subscription: We have subscriptions or plans for DTH, Broadband, mobile recharge, etc. In the same way, Microsoft provides subscriptions to use the resources with mainly 2 types pay-as-you-go model and a Free edition(it has limited resources). There are many sub-divisions under pay-as-you-go subscription.
- Azure Key Vaults: Key Vaults play a major role in safeguarding passwords. Key vaults contain secret names in which the passwords are stored. Only the secret name is exposed and not the password.
- Azure Storage Accounts: Storage accounts are created in order to hold all the Data storage-related resources like Azure blobs, Containers, File shares, Queues.
- Azure Linked Services: These services are used like connection strings that connect to external resources.
Problem Statement
Let us consider, we need to deploy activities(here copy activity from blob to blob), from the Dev data factory to the Prod data factory.
Prerequisite: Need to create a Storage account, Db, SQL server, connection strings, key vaults, secrets, etc, for both dev and prod.
Implementation
Follow the below steps. to use an Azure Manager:
Step 1: The below image shows there are no activities in the production data factory.
iNo activity in prod data factory
The below image shows the copy activity present in the dev data factory, that needs to be deployed in the production data factory.
Simple copy activity in dev data factory
As in the below figure, click the manage icon inside the dev data factory, then click on the ARM template blade over there. If you click on the Export, the ARM template downloads in .zip format.
Export ARM template
Step 2: After extracting the zip file, we can see these four files inside it. We can make the changes if needed in the arm_template(3rd one from the top).
Step 3: From the prod data factory click on manage -> ARM template -> Import -> Build your own template in the editor(In Custom deployment window) -> Load file -> select the above arm_template file from your local machine -> click save.
Step 4:- In the next window, provide all the prod-related configurations like subscription, resource group, key vaults, region, etc.. Taking this configuration as a reference, the prod configurations reflect while deployment. Then click on review+create -> create -> it starts deploying. Finally, we can see the copy activity(as in the fig below) in the prod DF.
Prod DF with copy activity
Finally, we have learned to use the ARM template. Based on the project, the ARM template usage approach changes.