Pre-requisite: Azure
A Self-Hosted Integration Runtime is a component of the Azure Integration Runtime (AIR) that enables data integration between on-premises systems and the Azure cloud. It is a secure, managed, and serverless computing infrastructure that allows you to connect your on-premises data sources, such as databases and file systems, with Azure data services.
Components of Self-Hosted Integration Runtime
On-Premises Gateway
- An on-premises gateway is a software application that provides secure, bi-directional data transfer between a local network and the cloud. It is used for self-hosted integration runtime, which is a fully managed, server-less Azure service that enables organizations to run data integration processes in the cloud.
- It provides a secure connection between on-premises data sources and cloud-based solutions, such as Azure Data Factory, Azure Logic Apps, and Azure Machine Learning. It also provides data transformation, data cleansing, and data movement capabilities.
- The gateway can be installed on the local network and configured to securely transfer data between on-premises data sources and cloud-based services. The gateway is responsible for encrypting data and securely transferring it to the cloud, as well as decrypting and securely transferring data from the cloud. It also provides authentication and authorization for ensuring secure data access to the cloud.
- On-premises gateways are pieces of software that act as a bridge between on-premises data sources and cloud-based services. They are used to securely connect corporate networks to cloud services, allowing for the transfer of data between the two.
- On-premises gateways are often used when an organization wants to keep its data and applications on-premises but still use cloud services. A self-hosted integration runtime is an instance of the Azure Integration Runtime (IR) that is installed and managed on-premises.
- It provides a secure connection between on-premises resources and the cloud, allowing for the transfer of data between the two. It is typically used when an organization wishes to keep its data and applications on-premises but still uses cloud services. Self-hosted integration run times are commonly used in hybrid cloud scenarios, where data is transferred between on-premises resources and cloud services.
Cloud Component
- A self-hosted integration runtime is a component of Azure Data Factory that allows users to execute data processing activities on-premises. It is used to perform activities such as data movement from on-premises data sources to cloud data stores. It also enables users to execute activities such as copying data from cloud data stores to on-premises data stores and executing data transformation activities such as running SSIS packages on-premises.
- The Self-Hosted Integration Runtime consists of an on-premises machine with an agent installed on it. This agent is responsible for managing and executing activities within the self-hosted integration runtime. The agent communicates with the Azure Data Factory service in the cloud to access the resources necessary to perform the activities.
- The Self-Hosted Integration Runtime also includes a cloud component. This component consists of a virtual machine in the cloud that is responsible for managing the activities. It also provides services such as logging, monitoring, and alerting. The cloud component is used to manage the self-hosted integration runtime and ensure that it is running optimally.
- It is used to move data between on-premises data stores and cloud data stores, and for executing data transformation activities such as Self-Hosted Integration Runtime provides a simple, secure, and reliable way to move data between on-premises systems and the Azure cloud. It is easy to deploy, configure, and manage, and it provides a secure and reliable platform for data integration. It also provides features such as scalability, fault tolerance, and automated deployment, allowing you to quickly and easily incorporate data integration into your applications.
Uses of Self-Hosted Integration Runtime
- It is a feature of Azure Data Factory (ADF) used to facilitate data movement and transformation between on-premises and cloud-based data stores.
- It is used when there is an on-premises data source that cannot be accessed directly from the cloud. The Self-hosted integration runtime is installed on an on-premises computer and acts as a bridge between the cloud and the on-premises data source.
- It can be used to move data between on-premises and cloud-based data stores and to perform transformations on the data as part of the data movement process.
- They are useful when the on-premises data source is not accessible through the internet, or when data latency needs to be minimized. Examples of scenarios where Self-hosted integration runtime is useful include moving data between an on-premises Hadoop cluster and Azure Data Lake Store, moving data between an on-premises SAP system and Azure Synapse Analytics, and moving data between an on-premises SQL Server database and Azure SQL Database.
- Azure Self-Hosted Integration Runtime (IR) is a secure, on-premises data integration service used to connect to on-premises resources. It can move data between cloud and on-premises data stores, such as SQL Server, Oracle, and others.
- Self-Hosted IR is ideal for organizations that need to securely access and move data between cloud and on-premises data stores without exposing their data to the public internet. It also provides a secure, point-to-point connection between on-premises resources and the cloud, which can help reduce latency and improve performance.
- Self-hosted IR is useful when organizations need to move data between cloud and on-premises data stores, such as hybrid data warehouses, and when there are security or compliance requirements that prevent them from connecting to the public internet. It can also be used for batch processing, streaming, and real-time data ingestion. Additionally, self-hosted IR can be used to move data between cloud and on-premises data stores as well as between cloud data stores.
Steps to Configure Self-Hosted Integration Runtime
Pre-requisites for SHIR
- A Physical or VM running in your on-premises data center or Windows Azure Virtual Machine.
- A Network Connection between the on-premises data center and the Azure environment.
- A Windows account with administrator privileges on the machine hosting the self-hosted integration runtime.
- A SQL Server instance or an Azure SQL Database instance.
- the most recent iteration of the Microsoft .NET Framework Microsoft Visual C++ Redistributable Package newest version.
- A supported version of Windows Server.
- A supported version of SQL Server.
- The latest version of the Azure Data Factory .NET SDKA local computer or server with hardware and software resources that meet the system requirements for the specific integration runtime.
Step 1: Log in to the Azure portal
Step 2: Search for Azure Data Factory
Step 3: Create and lunch the Azure Data Factory
Step 4: After launching Azure Data Factory go to the manage tab and go to Integration Runtime
Step 5: Go to the manage tab and go to Integration Runtime click on + New
Step 6: After clicking on New then select Self-Hosted
Step 7: Specify the name of the Integration Runtime and click on create
Step 8: After the creation of IR open it and copy key 1 or key 2
Step 9: Paste that key into Installed Microsoft Integration Runtime On your PC
Step 10: After pasting the key click on Register