Introduction

The objective is to learn terraform and configure terraform with AWS provider.

Terraform

The terraform { } block includes the version of terraform, providers/dependencies settings, and the backend configuration.

required_providers

The required_providers {} contains list of provider settings. Each provider includes:

  • source: source will include a optional HOSTNAME, a NAMESPACE and a TYPE. So the complete definition for source is registry.terraform.io/hashicorp/aws.
  • version: the version number in string literal. The version can be a composite of many conditions, seperated by comma:
    • “=”: allow only the version specified
    • “!=”: exclude the version
    • “>, >=, <, <=”: specify the comparison against a specific version
    • “>”: allow all patch version greater than the specified version, so “> 1.2.1” allows all “1.2.2” and “1.2.4”, but not “1.2.0”
  • local name: local name is the unique identifier (allowed one per module) that refers to its provider configuration. Conventionally, terraform suggests local name should be the prefix name of its resource types. For example, here, prvoder hashicorp/aws uses a local name “aws” that has resource types such as aws_instance, aws_sns_topic etc.
    • note: the official docs is wrong for the naming convention. Underscores are not allowed based on this issue.”aws_profile” is not allow, but “aws-profile” is ok.

backend configuration

The backend "name" {} controls the save location (.tfstate file) of Terraform managed resource state. By default, Terraform uses backend "local" {} which saves the file in the same directory the command is executed.

For a cloud environment, because the .tfstate file shouldn’t be checked out to the version control, terraform needs a remote storage for its state. And to ease the process, for example, to configure a backend using S3, do

1
2
3
4
5
6
7
8
9
terraform {
    backend "s3" {
        bucket = "bucketName"
        key = "filename

        # specify credential such as profile, accesskey etc.
        # that has access to S3 bucket.
    }
}

provider

The provider "aws" {} block defines neccessary metadata for target providers. For example, for the hashicorp/aws provider, it needs to configure target region, credential profiles, access key etc.

Alias

provider block can use a alias value to define multiple provider profile for the provider. For example

1
2
3
4
5
6
7
8
9
10
11
12
13
14
provider "aws" {
  alias  = "west"
  region = "us-west-1"
}
provider "aws" {
  alias  = "east"
  region = "us-east-1"
}
resource "aws_instance" "instance1" {
    provider = aws.west
}
resource "aws_instance" "instance2" {
    provider = aws.east
}

will deploy two ec2 instance in two regions using seperate aws provider.

Resource

The resource {} block represents a single unit of resource depolyment. I would consider this as an interface/abstraction that terraform API provided to manage one resource.

For example, resource "aws_s3_bucket" "bucket1" {} is a abstraction representing a S3 bucket. “bucket1” is a local name in Terraform that other resources can reference using aws.aws_s3_bucket.bucket1.

Module

The purpose of module block is to create reusability of complex resources. Based on the official docs and this reddit post, one directory (not including its subdirectory) is considered as one module. Just like in a programming language, you would refactor a complex process into a seperate function and only expose neccessary parameters. In Terraform, you can put the cohesive resources into a seperate directory. For example, with the following file structure,

1
2
3
4
5
6
7
8
9
10
root
    |_ lambda/
        |_ main.tf
        |_ variables.tf
    |_ instances/
        |_ main.tf
        |_ variables.tf
    |_ main.tf        # contains resource definition
    |_ variables.tf   # contains constant and input variables
    |_ output.tf      # contains output definition

I have all resource definition related to lambda and ec2 instances in sperate folders. root/main.tf can access all resources/variables from root/variables.tf and root/output.tf, but not root/lambda/main.tf and root/instances/main.tf. In order to bring those two modules into root module’s scope, you need to use a source definition with relative pathing and supplied with required variables from root/lambda/variables.tf and root/instances/variables.tf. 

1
2
3
4
5
6
module "module_name" {
    source = "./lambda"     # or "./instances"

    param1 = ...
    param2 = ...
}

Notes

This section will be some notes

Backend configs

Because Terraform does not allow variables within the backend block, e.g. the following will result an error

1
2
3
4
backend "s3" { 
    accessKey="${var.accessKey}"
    ...
}

The partial configuration suggests to use terraform apply cli with -backend-config= option. For instance, if running Terraform with Github Actions, I can configure backend with cli

1
2
3
4
5
terraform apply -backend-config="region=$region" \
                -backend-config="bucket=$bucket_name" \
                -backend-config="key=$key" \
                -backend-config="accessKey=$access_key" \
                -backend-config="secret_key=$secret_key"

Lambda with SNS trigger

To configure a SNS trigger for AWS Lambda:

  1. Define both resources for SNS topic and lambda function
  2. the SNS need to define a topic subscription to the lambda function
  3. and the lambda function needs to define invoke permission to allow SNS resource to invoke the function.

For example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
# Create sns topic and lambda
resource "aws_sns_topic" "sns_topic" {
    ...
}

resource "aws_lambda_function" "lambda_function" {
    ...
}

# Create SNS subscription
resource "aws_sns_topic_subscription" "sns_lambda_subscription" {
  depends_on = [aws_sns_topic.sns_topic, aws_lambda_function.lambda_function]

  topic_arn  = aws_sns_topic.sns_topic.arn
  protocol   = "lambda"
  endpoint   = aws_lambda_function.lambda_function.arn
}

# Create lambda permssion
resource "aws_lambda_permission" "with_sns" {
  depends_on = [aws_sns_topic.sns_topic, aws_lambda_function.lambda_function]

  statement_id  = "AllowExecutionFromSNS"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.lambda_function.function_name
  principal     = "sns.amazonaws.com"
  source_arn    = aws_sns_topic.sns_topic.arn
}

One thing also to mention that the “aws_sns_topic_subscription” resource uses default filter_policy that applies to MessageAttribute, not MessageBody. The change the value, set

1
2
3
4
5
resource "aws_sns_topic_subscription" "sns_lambda_subscription" {
    ...
    filter_policy_scope = "MessageBody"
    filter_policy=...
}