Infrastructure As Code with Terraform

What is Infrastructure as Code?

Infrastructure as Code (IaC) is a term to describe a paradigm change where operators stop to create, configure and change the infrastructure manually and begin to automate everything using code. Exactly like developers create an application, the operators will create, update, delete the Infrastructure using code. The Infrastructure as Code allows to control everything by code like compute, storage, network to finally datacentre management.

Why use Infrastructure as Code?

There are several benefits to use IaC and the majority are provided by using code to describe the target infrastructure. The code can be duplicated quickly for reproducibility and speed. It can be stored under version control system for versioning (reviewing, rollback), auditing, collaboration and sharing. It enables configuring  CI/CD pipeline – like other development code – to improve the delivery of the resources (quality, speed), to help on continuous testing (security, bug, etc…) and to accelerate the evolution. Every information inside the code becoming a kind of documentation, where of course, comments are still necessary.

The IaC allows to automate all tasks and to limit the risks inherent to human actions, accelerate them and help the operators to save time and work for more added-value activities. Due to automation, it pushes towards immutable infrastructure. You recreate from code rather than restore, update or reconfigure. The disaster recovery can be automated. The IaC also helps to use rules, policies, and structures on your infrastructure.

All the advantages of IaC make it indispensable for the Dev Ops activites.

Why only now?

The IaC principle exists since several years but was limited by lack of existing standards among editors/constructors. Some hardware was also incompatible to interact by code. A lot of Datacenter tasks required manual actions. With the new standard like REST API and the Software-Defined Infrastructure (SDDatacenter, SDNetwork, SDStorage) approach by constructors, the IaC became an area of interest even still complex and costly. The arrival of the Cloud Computing and Kubernetes have changed the game! Today on Kubernetes and with the Cloud Services Provider (CSP), everything can be managed by code easily using APIs. Working without IaC on the GCP and Azure would be a mistake, not benefiting fully of all the advantages of the Cloud.

What type of tools used to make Infrastructure as Code?

There are several types of tools, languages to use Infrastructure as Code.

First, we have Ad Hoc scripts with languages like Python, Powershell, Bash, Shell, etc… It is very flexible, with a lot of library available, but difficult to use and maintain for complex configurations.

Below, a small example to compare the tools to install apache, retrieve code sources and start the web server.

sudo apt-get update
sudo apt-get install –y apache2 php
sudo git clone https://github.com/la-redoute/myapp.git  /var/www/html/app
sudo systemctl start apache

There are configuration management tools like Ansible, Chef, DSC and Puppet. These tools permit to manage the configuration with a big advantage compared to Ad Hoc scripts, because they bring code convention, ready to use module, idempotence and distribution.

Same example than above in Ansible:

- name: Update the apt-get cache
   apt:
       update_cache:  yes
- name: Install PHP
   apt:
       name: php
- name: Install Apache
   apt:
       name: apache2
- name: Copy the code
   git: repo=https://github.com/la-redoute/myapp.git dest=/var/www/html/app
- name: Start Apache
   service: name=apache2 state=started enable=yes

Other type of IaC is Server templating tools like Docker, Vagrant and Packer. Rather than manage some configuration after deployment, these tools enable to create, manage the image that will be used for the deployment.

Same example than above in Packer:

{
"builders": [{
     "type":"googlecompute",
     "account_file":"account.json",
     "project_id":"my project",
     "source_image":"debian-7-wheezy-v20150127",
     "ssh_username":"packer",
     "zone":"us-central1-a"
    }]

"provisionners": [{
       "type":"shell",
       "inline": [
           "sudo apt-get update",
           "sudo apt-get install –y php",
           "sudo apt-get install –y apache2",
           "sudo git clone https://github.com/la-redoute/myapp.git /var/www/html/app",
           "sudo systemctl start apache2" è "sudo systemctl enable apache2"
        ]
    }],
}

Of course, no need to start apache because it will start automatically after the image deployment.

Finally, there are Provisioning Tools like Terraform, Azure Resource Manager (ARM) and Google Cloud Deployment Manager (DM). These tools allow to deploy all the Datacenter infrastructure (Compute, Storage and Network) very easily and quickly. Provisioning tools are designed for the Cloud.

What are the different approaches?

To use IaC, there are two approaches very coupled with the tool used. We can code the infrastructure using declarative languages, or procedural languages.

In a procedural language, we need to define the whole process to obtain the result expected, and if you rerun the code, that will make twice the same action. For example, if you decide to create a VM, if you run your code twice, that will create two new VMs.

In a declarative language, we need to define the expected target and the tool will provide the actions in order to obtain this result. If you rerun your code a second time, nothing will be provided because already done during the first run. For example, if you decide to create a VM, if you run your code twice, that will create only one VM. During the second run, the tool will only check if the VM exist as declared in your code. It’s the idempotence.

A third approach exists and will be the next generation of IaC. It’s the Artificial Intelligence approach that will permit to know the desired state before executing the changes to consider the application, the impacts, etc.

What is Terraform and why La Redoute use it for provisioning?

Terraform is an open source infrastructure provisioning software developed by Hashicorp in 2014. It is developed using the GO language. Terraform does not require a server to run because it uses APIs. It uses simple language HCL (Hashicorp Common Language) or Json to declare the infrastructure you want to deploy.

It’s clearly a declarative language able to handle dependencies, order and duration between resources when configuring a complex infrastructure. It is compatible with a very long list of providers (here). You can for example create your infrastructure for a variety of CSPs (GCP, Azure, AWS, …), for containers orchestrator (kubernetes, rancher, …), for V-Sphere, for SaaS solution (Datadog, Gitlab, Fastly,…) and for legacy infrastructure (Cisco, F5,..).

La Redoute has chosen to use Terraform to manage CSP because compared to other type of IaC tools. Terraform allows to quickly  create infrastructure. If we would use configuration tools like Puppet – that would be possible – it would be more complex and even worse with python.

We can also use the tools provided by the CSP but they are only compatible with their platform. Terraform, even if it’s necessary to adapt the code for each provider, propose the same agnostic language. If you understand the language (and at least the CSP) you can easily code your infrastructure. Another advantage of Terraform, it’s an open source tool. It means that lot of people contributes to its evolution and that the tool could be used without expensive cost. Know that Hashicorp propose an enterprise solution with support.

How to conclude?

Today using Cloud services without using IaC is not realistic. Terraform provides so many advantages to configure infrastructure in CSP, but it does not replace everything. That must be used in collaboration with other tools like Packer (to construct the image before deploying), Puppet (to configure your application after deploying), Helm (to deploy application inside Kubernetes) or scripts like python, shell (to run some jobs).

IaC requests ops people to deal with code and manage it well from the beginning using GIT and frameworks. Also that means that ops people will need training on those tools.

Even if tools are important, do not forget that the collaboration between dev, sec, network and ops Teams is essential (even more when using IaC). So, this paradigm change requires the teams, organization and process aligned as a key factor of success.