Once you have some Terraform configuration written within a directory, you can run CLI commands to provision your infrastructure with Terraform. The combination of these commands to provision, change, and delete remote cloud resources via Terraform is referred to as a Terraform workflow.

Terraform Workflow Commands

terraform init: The first time these commands are run within your directory, you should run the terraform init command. This initializes Terraform to run in that directory. It creates a state file if your Terraform state backend is a local file, and downloads binaries for any specified provider versions.

terraform plan: The next (or first) command to run in a Terraform workflow is the terraform plan command. Terraform essentially checks what changes would be made to your cloud environment (either created, modified, or deleted) and outputs that information. If your organization does not have a robust drift detection and mitigation strategy, you may discover infrastructure drift during this step in the workflow.

terraform apply: After running terraform plan, and confirming that the changes are expected and as desired, then terraform apply is run. This command will actually make changes to your cloud environment, creating, modifying, or deleting resources and updating your Terraform state file to reflect the changes found in your Terraform configuration. This command may take several minutes to run, as the resources being edited in your cloud may take several minutes to occur. If your changes are taking longer than 15 minutes to run, chances are high that you are trying to make too many changes within a single Terraform state file. More on state files below.

Options for Running Your Workflow

The Terraform CLI workflow of init, plan, and apply are obviously quite simple commands in and of themself. Of the main components, they are the easiest to write, but at the same time can be tricky to manage in practice within an organization.

Local Execution

Running your Terraform workflow locally is the easiest to get started with, but doing so does not scale well beyond a single developer managing all of your organization’s Terraform. To run your workflow locally, you simply need to install a version of Terraform on your local machine, and execute the CLI commands within the equivalent of Mac’s terminal or Window’s Powershell. For 99% of organizations, however, this does not scale. For example:

– How do we ensure that our Terraform workflows are running against a single source of truth?

– How as an organization do we ensure only approved Terraform is being applied against our remote cloud environments?

– How do we automate workflow runs?

With local execution, any developer can change Terraform configuration locally and run Terraform workflows. This means any developer can use an extremely powerful IaC tool to change, create, and delete remote cloud resources without checking against an approved, single source of truth. All of which is usually untenable for anything outside of a personal project.

Custom CI/CD Pipeline

The other way to run our Terraform workflows is to do so through a custom CI/CD pipeline within your orchestration tool of choice when new Terraform configuration has been merged into key branches. GitHub Actions, GitLab CI, and CircleCI all support running Terraform workflows.

This method can be free outside of what you need to pay your CI/CD orchestration tool, but can be a bit tricky to manage if you want to handle concurrent workflow runs against a single state file, or as you scale to multiple state files.

For example, if two developers closely open pull requests that would affect Terraform configuration on the same state file, the second pull request may simply fail given the first pull request has locked the Terraform state file. While this is good behavior for Terraform, it creates confusion on your dev team if these edge cases are not handled correctly, which can be a decent amount of work to build yourself. Lastly, variable management can be a bit funky; needing to keep your variables as secrets within your CI/CD pipeline vs. a secret manager within your cloud provider can be a risky component for organizations as they scale.

Atlantis

Atlantis is an open-sourced, self-hosted application that allows you to run your Terraform in a more integrated manner with your Pull Requests. Specifically, when a Terraform pull-request related to Terraform is opened, Atlantis automatically runs terraform plan and outputs the results as comments on your pull request. Upon approval and merge, Atlantis will run terraform apply etc.

When compared to the custom CI/CD pipeline, Atlantis has several advantages. Namely, a more easily visible Terraform workflow saved within your Pull Requests as opposed to being buried within your CI/CD platform’s logs. For tracking Terraform-driven changes over time, and making Pull Request reviews accessible to more developers, this is a significant benefit.

Of course, the trade off is that it is an open-sourced package that is not trivial to set-up, and may be more difficult to maintain than a custom CI/CD pipeline running natively on GitHub Actions or the like.

Terraform Cloud and Terraform Enterprise

Terraform Cloud is HashiCorp’s cloud-hosted, multi-tenant, Terraform management platform. 

It is an excellent tool for running Terraform workflows on git hooks, and being a SaaS offering, all of the heavy lifting and support required by an Atlantis solution is taking care of for you. Terraform Cloud can also, for the right price, layer on additional functionality and checks to your Terraform workflow, like infrastructure policy enforcement, cost estimation, and infrastructure health checks. Further checks can be run through third party offerings via Run Tasks.

Terraform Cloud has a generous free tier for vanilla Terraform workflows; that being said, at scale with all features enabled, Terraform Cloud gets expensive quickly. Terraform Enterprise is simply a self-hosted version of Terraform Cloud for organizations with more stringent security requirements, and is a rather costly solution. 

Spacelift

Spacelift is a competing SaaS product to Terraform Cloud and Enterprise that also offers cloud-hosted and self-hosted versions, along with escalating capabilities beyond vanilla workflows by pricing tier. Unlike Terraform Cloud and Enterprise, Spacelift also offers support for other IaC and orchestration tools like Pulumi, Cloud Formation, and Ansible. They have their own discussion as to why they may or may not be a better fit for your use case than Terraform Cloud or Terraform Enterprise.

Choosing the Right Terraform Workflow for You

We’ve gone over five major Terraform workflow hosting options. Now its up to you to determine which option is the right one for your organization.

1) Small team looking to run vanilla workflows and nothing more? Maybe the native CI/CD pipeline methods or Terraform Cloud’s free tier is the right option.

2) Small to medium sized team running vanilla workflows and have engineering capacity to manage extra infrastructure? Atlantis might be a good option to look into for your team.

3) Planning on growing as an organization? We would recommend starting with the free tiers of a managed, cloud-hosted solution like Terraform Cloud or Spacelift and scale as needed from there.

4) Large scale organization needing robust security and other checks within your workflow? You probably will do well with a self-hosted SaaS offering like Terraform Enterprise or self-hosted Spacelift.

There you have it, five ways to run your Terraform workflows. Think we missed anything? Drop us a line!

dragondrop.cloud’s mission is to automate developer best practices while working with Infrastructure as Code. Our flagship product regularly scans and identifies resource changes that have occurred outside of a Terraform workflow (e.g. drift) so that dev teams can have a Cloud environment that is fully represented as code. All of our tools are self-hosted by our customers, with no data ever leaving their servers. To learn more, schedule a demo or get started today!

Your content goes here. Edit or remove this text inline or in the module Content settings. You can also style every aspect of this content in the module Design settings and even apply custom CSS to this text in the module Advanced settings.

Learn More About Terraform