What is Terraform? How Does Terraform Work?

Feb 7, 2023Terraform

What is Terraform?

Terraform is the leading Infrastructure as Code (IaC) tool (see our article for a review of IaC). It is fully open-sourced, and managed by HashiCorp. Over 1000+ different infrastructure providers can be controlled via Terraform, and new providers are being created all of the time. This large, continuously growing ecosystem makes Terraform extremely powerful across a wide range of use cases. As a tool, Terraform configuration is written in either HCL or JSON, and the Terraform binary is generally run through the CLI.

Using Terraform, organizations can create, version, change, and delete infrastructure for virtually the entire cloud footprint. Pretty cool, right? But how does this work in practice?

How Does Terraform Work?

Like any large open source project that has matured over many years, Terraform has a great deal of features and potential complexity. Our aim with this post is not to replicate Terraform’s already excellent technical documentation; rather, we want to provide accessible, high-level information on the main components of Terraform and how they fit together. The former is great for nitty-gritty implementation. The latter, however, is essential for understanding how Terraform should broadly work within your organization.

At its core, Terraform allows you to declaratively write infrastructure configuration as code, and then when running a Terraform workflow of CLI commands, interact with remote cloud provider APIs to provision, change, or delete that infrastructure.

This birds-eye view is great in the abstract, but how does one actually operationalize the usage of Terraform within their organization? There are four major components of Terraform to keep in mind when developing your IaC strategy: The Configuration Language, CLI Commands, Terraform State, and Variable Management. We unpack each of the components below.

    The Terraform Configuration Language

    On a day-to-day basis, most interaction with Terraform is through the declarative HCL-based configuration language. This is where code is written that corresponds to resource definitions in the cloud. The syntax for HCL is fairly simple once used to it, but Terraform can also be written using formatted JSON and programming languages via the Terraform CDK. Below we show the generic structure of HCL within Terraform, as well as a real-world example with GCP.

    HCL Syntax Example

    There are several different outer-most block types, including resource, output, variable, module, terraform, provider, and data. The block type most often used is the resource type. This is where any server, database, or other Terraform provider’s cloud resource is configured. Out of the four major components, Terraform configuration takes the most time to write, but the least amount of time to “host” – this is almost always done within a Git-based Version Control System (VCS). We go into more detail with our Terraform Configuration Quickstart.

    Terraform CLI Commands

    Once you have some Terraform configuration written within a directory, you can run CLI commands to provision your infrastructure with Terraform. Note: the minimum necessary configuration must include a terraform block, and for a practical minimum configuration, a resource block.

    The first time these commands are run within your directory, you should run the terraform init command. This initializes Terraform to run in that directory. It creates a state file if your Terraform state backend is a local file, and downloads binaries for any specified provider versions.

    The next (or first) command to run in a Terraform workflow is the terraform plan command. Terraform essentially checks what changes would be made to your cloud environment (either created, modified, or deleted) and outputs that information. If your organization does not have a robust drift detection and mitigation strategy, you may discover infrastructure drift during this step in the workflow.

    After running terraform plan, and confirming that the changes are expected and as desired, then terraform apply is run. This command will actually make changes to your cloud environment, creating, modifying, or deleting resources and updating your Terraform state file to reflect the changes. This command may take several minutes to run, as the resources being edited in your cloud may take several minutes to occur. If your changes are taking longer than 15 minutes to run, chances are high that you are trying to make too many changes within a single Terraform state file. More on state files below.

    The Terraform CLI workflow of initplan, and apply are obviously quite simple commands in and of themself. Of the main components, they are the easiest to write, but at the same time can be tricky to manage in practice within an organization. Everyone on your team running CLI commands locally does not scale securely passed one developer.

    State File Management

    State Files can be thought of functionally as a snapshot recording of what remote cloud resources controlled by Terraform look like at a given point in time in the cloud. That is, state files maintain the “state” of remote resources controlled by Terraform. Terraform state also serves as a mapping between a given block of Terraform Configuration and the corresponding remote cloud resource controlled by that configuration.

    When terraform plan is run, Terraform refreshes the state file to reflect what is in your cloud, and then compares your configuration code against what is in the state file. 

    State is by default stored in a local file-system, but once your team size eclipses 1, this becomes an impractical solution.  Similarly, small infrastructure sandbox projects can be run safely using just one state file. But, as cloud footprints increase in size, and we wish to support production levels of availability, this becomes untenable. It is a best practice to segment Terraform infrastructure by environment and application area. This serves two purposes: 1) on a day to day basis you have much faster plan and apply runs, and 2) you create blast radiuses around your state files for more robust infrastructure. As an example of the later, if we have all of our Terraform infrastructure managed by one state file, and somehow corrupt said state file while updating an S3 bucket in our development environment, this can have a serious negative impact on our control over production networking resources. With separate state files by application and environment, such an error would only have negative outcomes for the Terraform resources managed by the persistent-storage state file.

    Choosing a shared state file management system is a very important decision and although not particularly difficult to set up, often requires a pre-mediated plan to avoid nightmarish scenarios of entire state file corruption or hour long terraform apply commands.


    Variable Management

    Using variables within your Terraform configuration allows you to more easily re-use Terraform configuration across different environments.

    To use variables within your Terraform configuration, you simply create a variable block, and reference it within another block (see example syntax below).

    There are other details around variable configuration, like sensitivity marking, that are outside the level of detail this article is meant to provide. These details can be found here.

    There are several ways to pass variables to your Terraform workflows, some of which are straightforward to employ when only a single user is touching your Terraform. Managing variables at scale and with proper security protocols is what makes this component of the Terraform stack worth spending extra time thinking about. We discuss several different approaches for managing variables and their pros and cons here.

    HCL Variable Syntax Example

    The Complete Terraform Stack

    This article is meant to give an understanding of what Terraform is, how it works, and how major components of the “Terraform stack” fit together. Given that, now you’ll need to make informed decisions about where your Terraform configuration will live, how your Terraform workflow commands will be run, what your State management strategy is, and how variables are securely injected into your Terraform workflow across different environments.

    Within this article we reference several other articles that help you make these informed decisions. For convenience, we consolidate these links below:

    1) 5 Ways to Run Your Terraform Workflows

    2) Quickstart: Terraform Configuration

    3) Managing Terraform Variables

    Think we missed anything? Drop us a line! 

    dragondrop.cloud’s mission is to automate developer best practices while working with Infrastructure as Code. Our flagship OSS product, cloud-concierge, allows developers to codify their cloud, detect drift, estimate cloud costs and security risks, and more — while delivering the results via a Pull Request. For enterprises running cloud-concierge at scale, we provide a management platform. To learn more, schedule a demo or get started today!

      Learn More About Terraform

      Everything Everywhere All as Code

      “Everything as Code” Definition Everything as Code is a philosophy for managing IT infrastructure where all components of infrastructure are created, managed, and deleted using code. This applies to container definitions, cloud infrastructure, on-premise server...

      read more

      Open Source driftctl Alternatives

      What is driftctl? driftctl is an OSS CLI tool that enables users to identify Terraform drift as well as unmanaged resources within a cloud environment. It is a quite popular tool and has collected over two thousand stars on GitHub. Why Would We Want a Replacement?...

      read more

      Why We Are Not Supporting OpenTF

      Background On August 10, HashiCorp changed the license to their previously “Open Source” projects to a Business Source License (BSL), making them now “source available” for all future releases. We discusssed in detail reasons and motivations for this change here. On...

      read more