Traditional system administrator teams used to manage multiple data centers and servers full-time. For this, they needed to provision and manage all resources in the data-center carefully and prepare plans to upgrade/migrate these resources to ensure that the business works smoothly.
Today, public cloud providers (AWS, Azure, and GCP) offer cloud infrastructure services to manage critical business using their own cloud data centers.
This cloud revaluation changes the way system administrators manage infrastructure resources. Today's engineers need to manage resources on cloud platforms as per business requirements, which makes correct configuration and security essential.
IaC can help in this situation, let’s learn more about it - Infrastructure-as-Code (IaC) is the process of provisioning and managing infrastructure resources by using code on Cloud platform. This code is in machine-readable templates, rather than physical configuration. IT infrastructure can fully provision directly from source code by using Open-source automation IaC tools which can be customized as per requirement. Being open-source it also eliminates any need for licensing cost.
Complex infrastructure can be spined-up, built-up, and scaled-up by using scripts. These resources are like - virtual machines, networks, firewall rules, routes, storage buckets, load balancers, and pre-configure databases. We can replicate these infrastructure resources quickly and easily during Development, Staging, and Production with pre-defined sets of parameters. Also, the environment can be quickly deployed in other regions to manage disaster recovery/backup scenarios.
Traditionally standard operating processes (SOPs) help to maintain infrastructure consistency. However, there is always a chance of encountering human errors, which may cause configuration differences and make the situation very difficult to debug. IaC tools help in standardization by using templates to minimize human errors and configuration differences. This will ensure the same configuration is deployed across environments thus making it easier to debug environment issues.
- Minimized Risk
Nowadays configurations need to be changed frequently to incorporate new features, integration, and performance improvement. It is very difficult to analyze the impact of changes and rollbacks in major releases. IaC tools have the feature that offers version control and maintains log records for each change which helps manage the impact in a better way.
In addition, these changes can help to rollback to the previous version with minimal impact on business. This process can also be automated using IaC tools to reduce risk in case of any issues or failures.
- Increased Efficiency
Environments can be spined-up and built-up based on refresh production data in a short time by using IaC scripts on multiple stages. Also, we can configure with Continuous Integration and Continuous Deployment (CI/CD) tools, which enables to reduce software development life cycle time. IaC tools scripts also manage to dismantle any orphan infrastructure environments.
- Cost savings
Infrastructure Automation using IaC tools allows system engineers to spend less time on deployment with minimal manual work. They can spend more time analyzing logs and auditing security vulnerability which helps to increase productivity. IaC tool scripts can also be used to wipe out unwanted resources, which saves costs on cloud platforms.
Microsoft provides IaC service - Azure Resource Manager (ARM). Using this service, users can provision infrastructure and control dependencies. The ARM templates manage resources by using JSON and allows you to declare multiple resources in one template for single project environments. The templates are reusable for an infinite number of times and provide similar results. The teams can monitor all builds/releases using the VSTS dashboard, and get overall quality of environments.
HashiCorp provides IaC tool Terraform for provisioning and managing cloud infrastructure. Terraform is an Idempotent and Cloud-Agnostic tool. It is also a multi-cloud capable tool, which equips it to automate specific infrastructure provisions to any platform supported by using the same set of terraform tools and languages. This helps in managing large & complex distributed applications across cloud platforms. It manages the remote state file and lifecycle of deployments, thus making it more advanced than ARM templates.
Terraform supports infrastructure-provisioning plan before deployment as well. By using a centralized file management code, terraform can replicate deployment in multiple environments with minimal code changes. Terraform supports Infrastructure resource creation workflow orchestration by using plan/apply cycle. It can run locally but needs to maintain a consistent working directory by using the wrapper script.
ARM Template Deployments using terraform:
Terraform does not manage any resources created by ARM templates, it can only manage the deployment. This means when you remove any resource from ARM template terraform, you will only remove reference of that resource in the state file and the resource will still be available in the Cloud.
Remote State Management on Azure:
There are two options to implement remote state management when targeting Microsoft Azure:
- Terraform Cloud
- Azure Blob Storage
Similar to providers, Terraform’s remote state management is based on a plugins architecture: for each project that you are working on, you will need to choose a remote state backend (provider) to use. Enabling version control for state files helps in controlling changes in infrastructure deployment.
Terratest is an open-source tool used to validate test conditions by using Go language with platform integration tests. Testing is a very important part of IaC because it is the only way to verify that the automatic process is working as per the logic written in code. There are different kinds of tests (integration/Smoke/Load/Security) to ensure infrastructure is provisioned as per the defined code parameters. This ensures that your application is secure enough. It is possible to automate testing of various scenarios by using Terratest, as it can integrate with terraform deployment before provisioning resources into production. Terratest deals with the entire Terraform plan, and executes Go code to test whether the infrastructure resource deployed is as per testing scenarios or not.
Continuous Integration using CI/CD Pipeline
Terraform code integrates with CI/CD pipeline to ensure infrastructure remains valid after every code change. It can be implemented easily with automation. These tools trigger pipelines after every Commit change is done in terraform code repository. Customized functions are also integrated with terraform code change (e.g. terratest, OPA, etc.), and infrastructure is deployed on the cloud environment after completing successful testing of all conditions and validations.
Validating Terraform Plans using Open Policy Agent
Open Policy Agent validates terraform plans before they are applied. It uses the Rego Query Language to express policies and generate evaluations based on specific inputs. It is challenging to test and validate changes without executing the code against a real prod/dev/stg environment. The feedback loop between writing a line of code and understanding it leaves its impact on the system.
E.g. Confirm Network Security Rule Does Not Allow RDP/SSH
Validate Daemonset Exists
Confirm Kubernetes Service Accounts
The terraform plan now passes all the Allow policies defined in the Open Policy Agent policy definition.
This blog explains the overview and use cases of IaC tools (Terraform, Azure Resource Manager, and Terratest & OPA) and frameworks to run on the Azure platform. These tools are easy to use and help to maintain infrastructure code in a secure & consistent format. These tools help in improving infrastructure provision processes (Accuracy/Speed/Security/Consistency/Reduce Risk/Increase Efficiency/Cost-Saving). Users can leverage features of above IaC tools that are right-fit for their environment and will make infrastructure deployment process effortless.