Simplify Terraform By Generating Configurations

Terraform is an awesome tool. To make it more awesome though we have wrapped it with some custom Ruby ERB templating to generate our terraform configurations from Yaml configurations.

Terraform uses a declarative language. You describe the state you want and it figures out how to get there. The declarative nature of Terraform does not afford us the same control that a language like Ruby can provide, which is fine, but I have found that I end up managing _massive_ Terraform configurations. It's easy to make mistakes. Generating our Terraform configurations allows us to create more robust services in a shorter amount of time. It's much easier to edit an 80 line yaml file than a 5000 line Terraform configuration. 

We have a script, let's call it "Terraform Generator", or `tg`. We pass to `tg` an environment configuration like `develop`. 

tg develop

This will generate the terraform configuration `terraform/develop/main.yml` from a yaml file at `configs/develop.yml`

The code to generate the file is pretty simple. By running the generated plan through `terraform fmt` we can also do some validation to ensure it's not broken! Simply put we read in the environments yaml file, parse an erb template with the content from that yaml file, output a terraform configuration file, then validate that file. 

`tg` does basically the following:

def output_terraform
    renderer = ERB.new(File.open("templates/#{@tf_template}").read)
    rendered = renderer.result(@context)
out_file = File.open("#{@output_directory}/main.tf", 'w')
out_file.puts(rendered)

format_terraform

end
def format_terraform
system “terraform fmt #{@output_directory}”
end
def create_output_directory
directory = “terraform/#{@config[‘config_src’]}”
unless File.directory?(directory)
  FileUtils.mkdir_p(directory)
end

directory

end
def main(*argv)
config_src = argv.last
@config           = YAML.load_file("configs/#{config_src}.yaml")
@tf_template      = 'main.tf.erb'
@output_directory = create_output_directory
@context          = binding

output_terraform

end
main(*ARGV)

The yaml file may look something like:

version: 1
region: us-west-1
tfstate: develop
cluster:
name: DEVELOP
load_balancers:
frontend:
ssl_certs:
- example.com
backend:
ssl_certs:
- example.com
ssl_certs:
default:
domain_name: ‘example.com’
subject_alternative_names:
- ‘*.example.com’
services:
my_service:
task_count: 2
cpu: 256
memory: 512
https:
load_balancer: frontend
url: my_service.example.com
health_check:
path: /
autoscaling:
min: 2
max: 8
cpu: 75
my_other_service:
task_count: 2
cpu: 256
memory: 512
https:
load_balancer: backend
url: my_other_service.example.com
health_check:
path: /
autoscaling:
min: 2
max: 4
cpu: 60

The ERB template:

terraform {
  required_version = ">= 0.14.5"
backend “s3” {
bucket = “terraform-state”
key    = “<%= @config[‘tfstate’] %>/terraform.tfstate”
region = “<%= @config[‘region’] %>”
# Force encryption
encrypt = true

}
required_providers {
aws = {
version = “~> 3.24.1”
}
}
}
provider “aws” {
region = “<%= @config[‘region’] %>”
}
<% @config[‘services’].each do |service, values| %>
…
<% end %>
… etc …