Terraform Deployments

Terraform allows infrastructure to be defined to deploy applications and other solutions as code and supports a plethora of on-premise and cloud deployment targets.

It is essentially based on building a graph of dependencies between resources, data and modules using the terraform language.

Due to the nature of the beast these things tend to run in the CI pipelines which makes editing these files frustrating as the edits have to be committed, pushed, runners have to be scheduled and usually the deploy pipeline is not the first job.

So good local tooling is needed to get fast feedback.

Terraform tooling on GUIX

In order to run terraform I need to first package it as it is not available in the GUIX repositories.

(define-public terraform
  (package
   (name "snam-terraform")
   (version "1.8.4")
   (source (origin
            (method url-fetch)
            (uri (string-append "https://releases.hashicorp.com/terraform/" version "/terraform_" version "_linux_amd64.zip"))
            (sha256
             (base32
	       "1i181cmzwlrx8d40z1spilcwgnhkzwalrg8822d23sqdmrs7a5hj"))))
   (build-system binary-build-system)
   (supported-systems '("x86_64-linux"))
   (arguments '(
      #:install-plan
      `(("." ("terraform") "bin/"))
      #:phases
      (modify-phases %standard-phases
		     ;; this is required because standard unpack expects
		     ;; the archive to contain a directory with everything inside it,
		     ;; while babashka's release .tar.gz only contains the `bb` binary.
		     (replace 'unpack
			      (lambda* (#:key inputs #:allow-other-keys)
				(system* (which "unzip")
					 (assoc-ref inputs "source"))
				#t)))))
   (inputs
    `(("libstdc++" ,(make-libstdc++ gcc))
      ("zlib" ,zlib)))
   (native-inputs
    `(("unzip" ,unzip)))
   (synopsis "A tool to describe and deploy infrastructure as code")
   (description
    "Terraform allows you to describe your complete infrastructure in the form of code. Even if your servers come from different providers such as AWS or Azure, Terraform helps you build and manage these resources in parallel across providers.")
   (home-page "https://hashicorp.com/terraform")
   (license #f)))

(define-public snam-terraform-1.6
  (package
    (inherit terraform)
    (version "1.6.6")
    (source (origin
             (method url-fetch)
             (uri (string-append "https://releases.hashicorp.com/terraform/" version "/terraform_" version "_linux_amd64.zip"))
             (sha256
              (base32
	       "002g0ypkkfqy5nf989jyk3m1l7l0455hsaq11xfhr5lbv4zqh5yi"))))))

I immediately added support to build older versions because that's what the customer is on and terraform is quite version dependent AFAICT.

Now I can create a manifest for this project. I usually bootstrap them with guile shell --export-manifes go gopls or similar and then add stuff when it comes up.

;; What follows is a "manifest" equivalent to the command line you gave.
;; You can store it in a file that you may then pass to any 'guix' command
;; that accepts a '--manifest' (or '-m') option.

(specifications->manifest
  (list "go" "gopls" 
        "google-cloud-sdk" 
        "postgresql" 
        "snam-terraform-1.6" ; from snamellit channel
        ))

Direnv support

In order to manage my project environment and align it with the CI environment I added the expected variables and use the guix support in the stdlib of direnv. This will create a guix environment configured from the manifest.

use guix

export DB_URL="postgresql://<db_ip>/myproj"
export DB_USER="xyz"
export DB_PASSWORD="secret"
export PGPASSWORD=$DB_PASSWORD

export VAULT_TOKEN="<blablabla>"
export APPTIO_URL=https://acme.tpondemand.com
export APPTIO_TOKEN=<blablabla>

export OPENAI_API_KEY=<blablabla>

export VAULT_ADDR=https://vault.acme.com
export STATE_BUCKET=com-acme-test-myproj-tf-state

export TF_VAR_project_short=myproj
export TF_VAR_project=com-acme-test-${TF_VAR_project_short}

PATH_add ./node_modules/.bin

Emacs support

Direnv Support

Emacs direnv mode will load the configuration from the .envrc file when opening a file in that project. The variables and apps are then available for complition, LSP, shell, etc.

;; enable direnv mode
(direnv-mode)

I just enable it globally because I want that always, not just for terraform.

Terraform Support

Enable some syntax highlighting and more importantly documentation help. Also set format-on-save and the indent to 2 spaces

;; configure terraform support
(require 'terraform-mode)
(add-hook 'terraform-mode-hook 
          (lambda ()
            (outline-minor-mode 1)))
(custom-set-variables
 '(terraform-indent-level 2)
 '(terraform-format-on-save t))

and expose the functionality in similar keybindings as I use for LSP support :

(evil-define-key 'normal terraform-mode-map
  (kbd "<leader>c k") #'terraform-open-doc
  (kbd "<leader>c f") #'terraform-format
  (kbd "<leader>c F") #'terraform-format-buffer
  (kbd "<leader>c n") 'flymake-goto-next-error
  (kbd "<leader>c p") 'flymake-goto-prev-error)

will now open a browser window with the documentation of the terraform element under the cursor. This does need terraform to be installed though.

Add support to Makefile

In order to save me from remembering the commandlines and because I keep the terraform files in a terraform directory I make some make targets to quickly access them.

tfinit: 
	terraform -chdir=terraform init -backend-config="bucket=${STATE_BUCKET}"

tfcheck:
	terraform -chdir=terraform validate

tfapply:
	terraform -chdir=terraform apply

tfplan:
	terraform -chdir=terraform plan

tflint:
	docker run --rm -v `pwd`/terraform:/data -t ghcr.io/terraform-linters/tflint

Workflow tips

Most of it is essentially hidden in the normal workflow. i.e. opening a terraform file will load it, saving formats it. The compile feature can be used to do file checking and linting.

Magit commit - push triggers the CI pipeline to run the deploy.

It is easy to test things out locally with the Makefile and it reduces the number of steps in the CI script , so less things which can do weird things.