Introduction

Today I’d like to share my experience of learning the basics of Terraform. I will show how I used it to set up AWS infrastructure to run the FastAPI backend of my notes app. Namely, I will be setting up an ECS cluster with one EC2 provider. I will also share some of the struggles I had to overcome - at least the ones I haven’t suppressed.

This post is part of a series where I create a basic CRUD app and improve upon the development and deployment of the app using DevOps tools and techniques.

Title
Creating a Notes App with FARM Stack
Containerize a FastAPI App with Docker
Deploying a FastAPI Container to AWS ECS
Setting Up GitHub Actions to Deploy to ECS
Using Terraform for ECS/EC2

Setup

First, I installed Terraform using the official guide from Hashicorp, with the following commands on my Linux workstation:

wget -O- https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
sudo apt update && sudo apt install terraform

Then, I created a main.tf file in a new terraform folder under my notes app folder, and added the following to define the AWS provider:

provider "aws" {
    region = "us-west-2"
}

Normally at this point one would need to configure AWS CLI authentication, which Terraform can use to communicate with the AWS API, but I had done this previously. You can read about it in my post - Deploying a Website to S3 Using AWS CLI.

From here, I ran terraform init to initialize the file structure and pull down provider requirements.

Adding Network Resources

Next, I added the resources for a basic network layout to main.tf:

provider "aws" {
    region = "us-west-2"
}

# VPC
resource "aws_vpc" "notes-prod-vpc" {
    cidr_block = "10.0.0.0/16"
}

# Internet Gateway
resource "aws_internet_gateway" "notes-prod-gw" {
    vpc_id = aws_vpc.notes-prod-vpc.id
}

# Route Table
resource "aws_route_table" "notes-prod-rt" {
    vpc_id = aws_vpc.notes-prod-vpc.id

    route {
        cidr_block = "0.0.0.0/0"
        gateway_id = aws_internet_gateway.notes-prod-gw.id
    }

    route {
        ipv6_cidr_block        = "::/0"
        gateway_id = aws_internet_gateway.notes-prod-gw.id
    }
}

# Subnet
resource "aws_subnet" "notes-prod-subnet" {
    vpc_id = aws_vpc.notes-prod-vpc.id
    cidr_block = "10.0.1.0/24"
    availability_zone = "us-west-2a"
    map_public_ip_on_launch = true
}

# Associate subnet with Route Table
resource "aws_route_table_association" "a" {
    subnet_id = aws_subnet.notes-prod-subnet.id
    route_table_id = aws_route_table.notes-prod-rt.id
}

# Security Group
resource "aws_security_group" "notes-prod-sg" {
    name        = "notes-prod-sg"
    description = "Allow inbound web traffic"
    vpc_id      = aws_vpc.notes-prod-vpc.id

    ingress {
        description = "HTTPS"
        from_port   = 443
        to_port     = 443
        protocol    = "tcp"
        cidr_blocks = ["0.0.0.0/0"]
    }

    ingress {
        description = "HTTP"
        from_port   = 80
        to_port     = 80
        protocol    = "tcp"
        cidr_blocks = ["0.0.0.0/0"]
    }

    ingress {
        description = "MongoDB_Atlas"
        from_port   = 27017
        to_port     = 27017
        protocol    = "tcp"
        cidr_blocks = ["0.0.0.0/0"]
    }

    ingress {
        description = "SSH"
        from_port   = 22
        to_port     = 22
        protocol    = "tcp"
        cidr_blocks = ["0.0.0.0/0"]
    }

    egress {
        from_port = 0
        to_port = 0
        protocol = "-1"
        cidr_blocks = ["0.0.0.0/0"]
    }
}

Then I ran terraform apply to create these resources and checked that they worked in the AWS console.

This part of the setup was pretty straightforward. The only part that tripped me up was later when everything was working except communication to the MongoDB Atlas database, which I quickly realized was because I hadn’t added a rule for it in the Security Group.

Defining an EC2 Instance

To set up the EC2 instance definition that would be the compute provider for my ECS cluster, I created a file named ec2.tf and added the following:

resource "aws_launch_template" "notes-prod-lt" {
    name_prefix = "notes-prod"
    image_id = "ami-0f53d557e55fc7064"
    instance_type = "t3.nano"
    key_name = "Key1"

    network_interfaces {
        security_groups = [aws_security_group.notes-prod-sg.id]
        subnet_id = aws_subnet.notes-prod-subnet.id
    }

    iam_instance_profile {
        name = "ecsInstanceRole"
    }

    block_device_mappings {
        device_name = "/dev/xvda"
        ebs {
            volume_type = "gp3"
            volume_size = 30
        }
    }   
    user_data = filebase64("${path.module}/ec2.sh")
}

A few things things to note:

  • image_id: This is an Amazon Linux image, which comes with Docker preinstalled.
  • key_name: This is an ssh key that I had created previously. It isn’t required to function, but it was incredibly valuable for troubleshooting.
  • iam_instance_profile: This is required and the role needs to be created from a predefined IAM role. I used the official documentation to set it up in the console.
  • user_data: This gives ECS the ability to deploy and run Docker contianers on our EC2 instance by definining an environment variable. It imports the file that I created, ec2.sh, which I added the following to:
#!/bin/bash
echo ECS_CLUSTER=notes-prod >> /etc/ecs/ecs.config

Next, I added an autoscaling group to ec2.tf, even though we’re not actually doing any scaling, because it is required for ECS.

resource "aws_autoscaling_group" "notes-prod-asg" {
    name = "notes-prod-asg"
    max_size = 1
    min_size = 1
    desired_capacity = 1
    launch_template {
        id = aws_launch_template.notes-prod-lt.id
    }
}

Setting up ECS

The last, and most difficult task, was to define the ECS infrastructure. To do that, I created a file named ecs.tf and started off by adding the cluster resource:

resource "aws_ecs_cluster" "notes-prod-ecs_cluster" {
  name = "notes-prod"
}

Then, I added the task definition resource. This does all of the heavy lifting:

resource "aws_ecs_task_definition" "notes-prod-ecs_td" {
  container_definitions = <<TASK_DEFINITION
[
  {
    "cpu": 0,
    "environment": [],
    "essential": true,
    "image": "584916250327.dkr.ecr.us-west-2.amazonaws.com/notes-api:latest",
    "name": "notes-api-container",
    "portMappings": [
      {
        "containerPort": 8000,
        "hostPort": 80,
        "name": "notes-api-container-8000-tcp",
        "protocol": "tcp"
      }
    ],
    "volumesFrom": [],
    "secrets": [
      {
        "name": "MONGO_USER",
        "valueFrom": "arn:aws:secretsmanager:us-west-2:584916250327:secret:notes-mongo-eIdNP9:MONGO_USER::"
      },
      {
        "name": "MONGO_PASS",
        "valueFrom": "arn:aws:secretsmanager:us-west-2:584916250327:secret:notes-mongo-eIdNP9:MONGO_PASS::"
      }
    ]
  }
]
  TASK_DEFINITION
  cpu = "1024"
  execution_role_arn = "arn:aws:iam::584916250327:role/ecsTaskExecutionRole"
  family = "notes-prod-task"
  memory = "64"
  requires_compatibilities = ["EC2"]
}

I spent many hours getting this part right. I love building automation, and one of the most challenging things to do when using automation tools that abstract the underlying processes is learning how to gain the visibility needed to be able to troubleshoot effectively. I had to learn where to look in Cloudwatch, the EC2 instance, and the Docker container.

Eventually, I found that I had assigned too much memory to the task, as it was in excess of the small amount of memory available to a t3.nano instance. This caused the task to fail, while giving no real error message in the AWS console.

The last resource needed in the ecs.tf file was the ECS service:

resource "aws_ecs_service" "notes-prod-ecs_service" {
  name = "notes-prod-ecs_service"
  cluster = aws_ecs_cluster.notes-prod-ecs_cluster.id
  task_definition = aws_ecs_task_definition.notes-prod-ecs_td.arn
  scheduling_strategy = "DAEMON"
}

Conclusion

This was a long, difficult, and rewarding project. It’s funny, because I don’t remember all of the different ways that I struggled to get this working. I think the sense of accomplishment overshadows the struggle. There’s something so gratifying and empowering to having code be the single source of truth for infrastructure and I’m proud of the effort it took to get there.

I’d like to expand on this infrastructure by adding a second EC2 instance to the cluster with a load balancer and perhaps play with auto scaling, but I might go down the Kubernetes rabbit hole instead. Either way, I look forward to the next challenge.

You can see the notes app in action at http://notes.rickt.io/ and the Github repo for it, including the Terraform code here

Cheers!
Rick

 

 

Resources