Terraform Issue

Walk Through A Terraform Issue

Terraform’s efficacy in managing infrastructure as code (IaC) has made its use for cloud resource deployment widespread. Teams can automate and optimize deployment processes that are customized to their requirements by combining Terraform with CI/CD pipelines like Jenkins. This guide will help solve Terraform Issue. Our team recently ran into a problem when Terraform’s Plan stage failed to parse a list variable, resulting in an unclear error message. This happened during a Jenkins pipeline run. The necessity of effective troubleshooting and problem-solving in preserving smooth automation workflows was brought to light by this disruption.

We started a methodical debugging procedure to solve this problem. To identify the primary cause of the parsing error, we made use of debugging tools included in Terraform and Jenkins. We found the discrepancy causing the failure by carefully going over our Terraform configurations and comprehending how variables were defined and passed. We worked together to find a solution to the parsing problem, and after several iterations of testing, we were able to successfully execute our CI/CD pipeline and strengthen our team’s expertise in handling intricate deployments using Terraform and Jenkins.

The Scenario

In Terraform we have an input variable defined as –

variable “sources” { type = list(string) }

and Jenkins file is written so that, user can provide the values as a comma separated string while deploying the resources. Then Jenkins is creating a list of sources from that comma separated string as –

def parse_source_ids(ids){ return ids.split(‘,’) }

and being passed in terraform plan like –

parsed_sources_ids = parse_source_ids(sources_ids) sh “””terraform plan -input=false -out plan \ -var=”environment=$environment” \ -var=”sources=$parsed_sources_ids” “””

This is an example command with actual value that ran-

terraform plan -input=false -out plan -var=environment=env -var=sources=[23e7ea8d-e82a-43eb-a208-f446128d5d9b, 562c4d0d-7538-40b8-a60e-8c0c1cf5c3b6]

Everything seems fine until it failed with an error message-

Terraform Error

What?

The error message is very confusing. Clearly the error message doesn’t match with what we are seeing. There is something weird happening in the middle.

Then we were looking at the terrraform plan command more carefully. We were thinking couple of things-

  • The value of environment was passed correctly
  • The values in sources list are also string(as value of environment was also not enclosed by quote and parsed successfully) and correctly separated by comma
  • What if we enclose the source ids with quote?
  • But they are already enclosed by quotes, aren’t they? ids.split(‘,’) should do that right?

Google to Rescue

After thinking random things it’s time for Goggling. It was clear that Terraform is not able to parse the value as list of string. So, Tried some keywords like-

  • How to pass a list of string to terraform plan as input variable
  • Can’t pass a list of string to terraform plan as input variable

Finally found a Git issue that describes the problem and solution-

In Unix shells, quotes are interpreted by the shell to override the default behavior of splitting arguments with whitespace, and so the first “level” of quotes is consumed by the shell and thus isn’t passed to Terraform at all. The first example terraform plan -var ‘myvar=[“zzzz”]’ works because the quotes prevent special interpretation of the inner quotes and so they are passed by the shell into the command line Terraform sees.

Bellow variations won’t work-

terraform plan -var ‘myvar=[‘zzzz’]’

terraform plan -var “myvar=[‘zzzz’]”

terraform plan -var myvar=[‘zzzz’]

In short this is a unix shell behavior that consumed the quotes around the source ids in the list and if we put single quote around the sources var like

var=’sources=$parsed_sources_ids’

and make sure double quotes are around the ids in the list then it should work.

Solution

Now we update the terraform plan command-

parsed_sources_ids = parse_source_ids(sources_ids) sh “””terraform plan -input=false -out plan \ -var=”environment=$environment” \ -var=’sources=$parsed_sources_ids'”””

and the input parser-

def parse_source_ids(ids){ return ids.split(‘,’).collect { it + “,” } }

It resolved the issue. Then a successful terraform command looked like-

terraform plan -input=false -out plan -var=environment=env -var=sources=[“b8798218-0f2e-4dfb-8e31-0f5db44ca893”]

In conclusion, using Terraform and Jenkins to deploy cloud resources has many benefits, including automation and scalability, but there are drawbacks as well that need to be carefully considered. Our most recent experience brought to light a recurring problem: using a Jenkins pipeline to parse a list variable in Terraform. The issue was caused by Unix shell behavior that unintentionally removed quotes from list elements, which led to an unexpected failure of the Terraform Plan stage.

We found a workable solution after conducting in-depth investigation and troubleshooting, including consulting internet resources. We successfully mitigated the parsing issue by enclosing the list elements in double quotes and the variable itself in single quotes within our Jenkins pipeline script. By using this method, we could be sure that Terraform carried out the plan precisely and that our deployment process remained dependable and honest.

When I think back on this experience, I see how crucial it is to have a solid understanding of tool interactions and conduct extensive testing within CI/CD pipelines. These kinds of problems offer great teaching opportunities and emphasize how important it is for DevOps workflows to have strong debugging procedures. By sharing our resolution process, I hope to help others who may be having similar problems and give them the tools they need to use Jenkins and Terraform to successfully navigate and resolve complexities in their own deployments. I appreciate your time, and I hope that this information will help your projects’ continuous integration and delivery run more smoothly and effectively.

Tadaaaa……….

Title Image credit – Jammin’ with Terraform, AWS & Jenkins

Picture of Shohag Mia

Shohag Mia

Software Development Engineer

Hire Exceptional Developers Quickly

Share this blog on

Hire Your Software Development Team

Let us help you pull out your hassle recruiting potential software engineers and get the ultimate result exceeding your needs.

Contact Us Directly

Address:

Plot # 272, Lane # 3 (Eastern Road) DOHS Baridhara, Dhaka 1206

Talk to Us
Scroll to Top