Infrastructure as Code with Terraform and GitLab (FREE)

Motivation

The Terraform integration features in GitLab enable your GitOps / Infrastructure-as-Code (IaC) workflows to tie into GitLab authentication and authorization. These features focus on lowering the barrier to entry for teams to adopt Terraform, collaborate effectively in GitLab, and support Terraform best practices.

Quick Start

Use the following .gitlab-ci.yml to set up a basic Terraform project integration for GitLab versions 14.0 and later:

include:
  - template: Terraform.gitlab-ci.yml

variables:
  # If not using GitLab's HTTP backend, remove this line and specify TF_HTTP_* variables
  TF_STATE_NAME: default
  TF_CACHE_KEY: default
  # If your terraform files are in a subdirectory, set TF_ROOT accordingly
  # TF_ROOT: terraform/production

This template includes some opinionated decisions, which you can override:

Including the latest GitLab Terraform Image.
Using the GitLab managed Terraform State as the Terraform state storage backend.
Creating four pipeline stages: init, validate, build, and deploy. These stages run the Terraform commands init, validate, plan, plan-json, and apply. The apply command only runs on the default branch.

This video from January 2021 walks you through all the GitLab Terraform integration features:

See the video: Terraform with GitLab.

GitLab Managed Terraform state

Terraform remote backends enable you to store the state file in a remote, shared store. GitLab uses the Terraform HTTP backend to securely store the state files in local storage (the default) or the remote store of your choice.

The GitLab managed Terraform state backend can store your Terraform state easily and securely. It spares you from setting up additional remote resources like Amazon S3 or Google Cloud Storage. Its features include:

Supporting encryption of the state file both in transit and at rest.
Locking and unlocking state.
Remote Terraform plan and apply execution.

Read more on setting up and using GitLab Managed Terraform states.

Terraform module registry

GitLab can be used as a Terraform module registry to create and publish Terraform modules to a private registry specific to your top-level namespace.

Terraform integration in Merge Requests

Collaborating around Infrastructure as Code (IaC) changes requires both code changes and expected infrastructure changes to be checked and approved. GitLab provides a solution to help collaboration around Terraform code changes and their expected effects using the Merge Request pages. This way users don't have to build custom tools or rely on 3rd party solutions to streamline their IaC workflows.

Read more on setting up and using the merge request integrations.

The GitLab Terraform provider

WARNING: The GitLab Terraform provider is released separately from GitLab. We are working on migrating the GitLab Terraform provider for GitLab.com.

You can use the GitLab Terraform provider to manage various aspects of GitLab using Terraform. The provider is an open source project, owned by GitLab, where everyone can contribute.

The documentation of the provider is available as part of the official Terraform provider documentations.

Create a new cluster through IaC (DEPRECATED)

Learn how to create a new cluster on Google Kubernetes Engine (GKE).

NOTE: The linked tutorial connects the cluster to GitLab through cluster certificates, and this method was deprecated in GitLab 14.5. You can still create a cluster through IaC and then connect it to GitLab through the Agent, the default and fully supported method to connect clusters to GitLab.

Troubleshooting

`gitlab_group_share_group` resources not detected when subgroup state is refreshed

The GitLab Terraform provider can fail to detect existing gitlab_group_share_group resources due to the issue "User with permissions cannot retrieve share_with_groups from the API". This results in an error when running terraform apply because Terraform attempts to recreate an existing resource.

For example, consider the following group/subgroup configuration:

parent-group
├── subgroup-A
└── subgroup-B

Where:

User user-1 creates parent-group, subgroup-A, and subgroup-B.
subgroup-A is shared with subgroup-B.
User terraform-user is member of parent-group with inherited owner access to both subgroups.

When the Terraform state is refreshed, the API query GET /groups/:subgroup-A_id issued by the provider does not return the details of subgroup-B in the shared_with_groups array. This leads to the error.

To workaround this issue, make sure to apply one of the following conditions:

The terraform-user creates all subgroup resources.
Grant Maintainer or Owner role to the terraform-user user on subgroup-B.
The terraform-user inherited access to subgroup-B and subgroup-B contains at least one project.

Invalid CI/CD syntax error when using the "latest" base template

On GitLab 14.2 and later, you might get a CI/CD syntax error when using the latest Base Terraform template:

include:
  - template: Terraform/Base.latest.gitlab-ci.yml

my-Terraform-job:
  extends: .init

The base template's jobs were renamed with better Terraform-specific names. To resolve the syntax error, you can:

Use the stable Terraform/Base.gitlab-ci.yml template, which has not changed.
Update your pipeline configuration to use the new job names in https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates/Terraform/Base.latest.gitlab-ci.yml. For example:
```
include:
  - template: Terraform/Base.latest.gitlab-ci.yml

my-Terraform-job:
  extends: .terraform:init  # The updated name.
```