Evolving an Application from a Single Server to a Modern Elastic Architecture P1: The Manual Process

Background

Cloud computing has significantly transformed how applications are developed and architected. Two of the most important changes that we've seen are:

Splitting applications into individual services known as microservices rather than a single large application
The ability to scale parts of an application horizontally automatically instead of scaling vertically

Microservices allow individual services to be updated independently, enabling dedicated development teams to focus on specific functionalities. This approach allows us to scale and update only the necessary services, rather than the entire application.

One key benefit of horizontal scaling is the ability to spin up new instances in minutes instead of hours, days, or weeks.. This allows us to keep up with the demand for our application and prevents outages increasing our availability and overall customer experience. Another benefit is that we can add these instances and remove them when we don't need them any more, this saves us money compared to scaling the instance vertically in which we increase the size of the instance instead of adding additional instances.

Along with the two above, another benefit of cloud computing is infrastructure as code (IaC). With IaC, we can define and deploy architecture programmatically, avoiding the need for manual configuration through the console. While the console might seem simpler for beginners, this project will demonstrate why IaC is a favorite among Cloud Engineers and Architects. In short, you can have your architecture in code, share that architecture with others, and spin up the exact same architecture in different regions or accounts in a matter of minutes.

In this project, I will be using Adrian Cantrill's lab on architecture evolution found here. If you are looking to learn more about AWS or are working on certifications I highly recommend checking out his courses. Building on his project, instead of clicking through the console, we'll be using Terraform to create our architecture.

Initial Terraform Code and Setup

For Part One of this project, we'll be using a Terraform template I created for the base architecture of our environment. Included in this will be the VPC, subnets, route tables, IGW, as well as security groups and rules. I've provided the code below, to use this, you'll first need to download the Terraform CLI for which the instructions can be found here. Once you download this, create a directory for your Terraform code and create a main.tf file using the code below. If you're using a Mac like I am, open terminal and go to the directory, you can then use "terraform init" to start up Terraform. Some things you will need as well is to export your AWS Access Key and Secret Access Key as environmental variables. This will allow for the services to be spun up in your account. As best practice you don't want to add these keys to your code, especially if you're going to be sharing them with others or uploading to a code repo. To create the env variables, you'll run

export AWS_ACCESS_KEY_ID="anaccesskey"
export AWS_SECRET_ACCESS_KEY="asecretkey"

# Configure AWS Provider
provider "aws" {
    region = "us-east-1"
}

# Retrieve the list of AZs in current AWS region
data "aws_availability_zones" "available" {}
data "aws_region" "current" {} 

# Define VPC
resource "aws_vpc" "vpc" {
    cidr_block = "10.16.0.0/16"
    enable_dns_support = true
    enable_dns_hostnames = true
}


# Create an Internet Gateway
resource "aws_internet_gateway" "igw" {

    vpc_id = aws_vpc.vpc.id
}

# Create the Public Route Table
resource "aws_route_table" "rtpub" {
    vpc_id = aws_vpc.vpc.id

    route {
        cidr_block = "0.0.0.0/0"
        gateway_id = aws_internet_gateway.igw.id
    }

    route {
        ipv6_cidr_block = "::/0"
        gateway_id = aws_internet_gateway.igw.id
    }
}

# Subnet Route Table Associations
resource "aws_route_table_association" "rt_assoc_pub_a" {
  subnet_id      = aws_subnet.pub-a.id
  route_table_id = aws_route_table.rtpub.id
}

resource "aws_route_table_association" "rt_assoc_pub_b" {
  subnet_id      = aws_subnet.pub-b.id
  route_table_id = aws_route_table.rtpub.id
}

resource "aws_route_table_association" "rt_assoc_pub_c" {
  subnet_id      = aws_subnet.pub-c.id
  route_table_id = aws_route_table.rtpub.id
}

resource "aws_subnet" "db-a" {
    vpc_id = aws_vpc.vpc.id
    availability_zone = data.aws_availability_zones.available.names[0]
    cidr_block = "10.16.16.0/20"
    map_public_ip_on_launch = true


    tags = {
        Name = "sn-db-A"
    }


}

resource "aws_subnet" "db-b" {
    vpc_id = aws_vpc.vpc.id
    availability_zone = data.aws_availability_zones.available.names[1]
    cidr_block = "10.16.80.0/20"
    map_public_ip_on_launch = true

    tags = {
        Name = "sn-db-B"
    }



}

resource "aws_subnet" "db-c" {
    vpc_id = aws_vpc.vpc.id
    availability_zone = data.aws_availability_zones.available.names[2]
    cidr_block = "10.16.144.0/20"
    map_public_ip_on_launch = true

    tags = {
        Name = "sn-db-C"
    }


}

resource "aws_subnet" "app-a" {
    vpc_id = aws_vpc.vpc.id
    availability_zone = data.aws_availability_zones.available.names[0]
    cidr_block = "10.16.32.0/20"
    map_public_ip_on_launch = true

    tags = {
        Name = "sn-app-A"
    }


}

resource "aws_subnet" "app-b" {
    vpc_id = aws_vpc.vpc.id
    availability_zone = data.aws_availability_zones.available.names[1]
    cidr_block = "10.16.96.0/20"
    map_public_ip_on_launch = true

    tags = {
        Name = "sn-app-B"
    }


}

resource "aws_subnet" "app-c" {
    vpc_id = aws_vpc.vpc.id
    availability_zone = data.aws_availability_zones.available.names[2]
    cidr_block = "10.16.160.0/20"
    map_public_ip_on_launch = true

    tags = {
        Name = "sn-app-C"
    }


}

resource "aws_subnet" "pub-a" {
    vpc_id = aws_vpc.vpc.id
    availability_zone = data.aws_availability_zones.available.names[0]
    cidr_block = "10.16.48.0/20"
    map_public_ip_on_launch = true

    tags = {
        Name = "sn-pub-A"
    }


}

resource "aws_subnet" "pub-b" {
    vpc_id = aws_vpc.vpc.id
    availability_zone = data.aws_availability_zones.available.names[1]
    cidr_block = "10.16.112.0/20"
    map_public_ip_on_launch = true

    tags = {
        Name = "sn-pub-B"
    }

}

resource "aws_subnet" "pub-c" {
    vpc_id = aws_vpc.vpc.id
    availability_zone = data.aws_availability_zones.available.names[2]
    cidr_block = "10.16.176.0/20"
    map_public_ip_on_launch = true

    tags = {
        Name = "sn-pub-C"
    }


}

# Creating Security Groups

resource "aws_security_group" "sg-wordpress" {
    name = "SGWordpress"
    description = "Controls access to Wordpress Instances"
    vpc_id = aws_vpc.vpc.id
}

resource "aws_vpc_security_group_ingress_rule" "allow_http" {
    security_group_id = aws_security_group.sg-wordpress.id
    cidr_ipv4 = "0.0.0.0/0"
    from_port = 80
    to_port = 80
    ip_protocol = "tcp"

}

resource "aws_vpc_security_group_ingress_rule" "allow_nfs_inbound" {
    security_group_id = aws_security_group.sg-wordpress.id
    referenced_security_group_id = aws_security_group.sg-efs.id
    from_port = 2049
    to_port = 2049
    ip_protocol = "tcp"

}

resource "aws_vpc_security_group_egress_rule" "allow_https_outbound" {
    security_group_id = aws_security_group.sg-wordpress.id
    cidr_ipv4 = "0.0.0.0/0"
    ip_protocol = "tcp"
    from_port = 443
    to_port = 443
}

resource "aws_vpc_security_group_egress_rule" "allow_http_outbound" {
    security_group_id = aws_security_group.sg-wordpress.id
    cidr_ipv4 = "0.0.0.0/0"
    ip_protocol = "tcp"
    from_port = 80
    to_port = 80
}

resource "aws_vpc_security_group_egress_rule" "allow_mysql_outbound" {
  security_group_id = aws_security_group.sg-wordpress.id
  cidr_ipv4 = "0.0.0.0/0"
  ip_protocol = "tcp"
  from_port = 3306
  to_port = 3306
}

resource "aws_vpc_security_group_egress_rule" "allow_nfs_outbound" {
  security_group_id = aws_security_group.sg-wordpress.id
  cidr_ipv4 = "0.0.0.0/0"
  ip_protocol = "tcp"
  from_port = 2049
  to_port = 2049
}

resource "aws_security_group" "sg-database" {
    name = "SGDatabase"
    description = "Controll access to Database"
    vpc_id = aws_vpc.vpc.id
}

resource "aws_vpc_security_group_ingress_rule" "allow_mysql" {
    security_group_id = aws_security_group.sg-database.id
    from_port = 3306
    to_port = 3306
    ip_protocol = "tcp"
    referenced_security_group_id = aws_security_group.sg-wordpress.id
}

resource "aws_security_group" "sg-loadbalancer" {
    name = "SG-LoadBalancer"
    description = "Control Access to Load Balancer"
    vpc_id = aws_vpc.vpc.id
}

resource "aws_vpc_security_group_ingress_rule" "allow_http_lb" {
    security_group_id = aws_security_group.sg-loadbalancer.id
    from_port = 80
    to_port = 80
    ip_protocol = "tcp"
    cidr_ipv4 = "0.0.0.0/0"
}

resource "aws_security_group" "sg-efs" {
    name = "SG-EFS"
    description = "Control Access to EFS"
    vpc_id = aws_vpc.vpc.id
}

resource "aws_vpc_security_group_ingress_rule" "allow_efs_in" {
    security_group_id = aws_security_group.sg-efs.id
    from_port = 2049
    to_port = 2049
    ip_protocol = "tcp"
    referenced_security_group_id = aws_security_group.sg-wordpress.id

}

resource "aws_vpc_security_group_egress_rule" "allow_efs_out" {
    security_group_id = aws_security_group.sg-efs.id
    ip_protocol = "-1"
    cidr_ipv4 = "0.0.0.0/0"

}

# iam role for wordpress
resource "aws_iam_role" "wordpress_role" {
  name = "WordpressRole"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect = "Allow"
      Principal = {
        Service = "ec2.amazonaws.com"
      }
      Action = "sts:AssumeRole"
    }]
  })

  managed_policy_arns = [
    "arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy",
    "arn:aws:iam::aws:policy/AmazonSSMFullAccess",
    "arn:aws:iam::aws:policy/AmazonElasticFileSystemClientFullAccess",
    "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
  ]
}

# iam instance profile
resource "aws_iam_instance_profile" "wordpress_instance_profile" {
  name = "WordpressInstanceProfile"
  path = "/"
  role = aws_iam_role.wordpress_role.name
}

Once you've created the environment variables as well as run "terraform init", you can next run "terraform plan" which will give you a plan of all the things that will be created in your account, and if this all looks good, we will finally run "terraform apply" which will actually create the resources.

For this we will be creating

A VPC with the CIDR range 10.16.0.0/16
An Internet Gateway which will route traffic to the outside internet
Three public, application, and database subnets
A route table to route the traffic from the public subnets to the public internet
Security groups for each subnet which will determine what kind of traffic is allowed in and out and from where
An IAM role for our Wordpress instance

This will likely take up to 10 minutes to create so nows a good time to take a little break. Once that's done we will start getting into creating our instance manually.

Manually Creating the Instance

To illustrate the advantages of IaC, we'll first create an instance manually, highlighting the challenges of this approach. For brevity, I’ve skipped creating the architecture manually, as it’s a lengthy process.

Creating Parameters

To begin, the first thing we will do is add some parameters to SSM store, this will store our parameters, so we can call them instead of hard coding them into our Terraform file. We will append this code to the existing Terraform file, save it, and then run "terraform apply" again to create the parameters.

# Create SSM Parameter Secrets
resource "aws_ssm_parameter" "DBuser" {
    name = "DBuser"
    type = "String"
    data_type = "text"
    value = "a4lwordpressuser"
}

resource "aws_ssm_parameter" "DBname" {
    name = "DBname"
    type = "String"
    data_type = "text"
    value = "a4lwordpressdb"
}


resource "aws_ssm_parameter" "DBpassword" {
    name = "DBpassword"
    type = "SecureString"
    data_type = "text"
    value = "4n1m4l54L1f3"
}

resource "aws_ssm_parameter" "DBrootpassword" {
    name = "DBrootpassword"
    type = "SecureString"
    data_type = "text"
    value = "4n1m4l54L1f3"
}

resource "aws_ssm_parameter" "DBendpoint" {
    name = "DBendpoint"
    type = "String"
    data_type = "text"
    value = "localhost"

This will create parameters that we will use when creating our Wordpress instance.

Launching the EC2 Instance

Now lets jump over to the EC2 console and click on "launch new instance". We're gonna name this instance "Wordpress-Full" and choose Amazon Linux as our image, and choose an instance size that's eligible for the free tier.

We won't choose a key pair for this project, since we'll be using Session Manager to connect to our instances. We will also be using the VPC we created and the subnet pub-A, auto-assign IP also needs to be enabled, and we'll use the Wordpress security group that we created in our Terraform file

For the instance IAM profile, we'll use the one create in our template as well. This will allow us to connect to our instance using Session Manager.

Click 'Create' to launch the instance. Once it's running and health checks are complete, select the instance and click 'Connect' at the top right.

Connecting To and Setting Up Our Instance

This will take us to the connection page, and in this we will use Session Manager. When we create the IAM role, we added the ability to connect using SSM, which allows for us to connect without a key pair.

Session Manager should be opened in a new tab, now we can use the "sudo bash" command, the "cd", and "clear". The next step will be to pull in the parameters we created earlier as store them as ENV variables. We will start with the DBPassword

- DBPassword=$(aws ssm get-parameters --region us-east-1 --names DBpassword --with-decryption --query Parameters[0].Value)

- DBPassword=`echo $DBPassword | sed -e 's/^"//' -e 's/"$//'`

The first command above pulls in the parameter value, and the second command removes the double quotes at the beginning and the end leaving us with only the value we stored as the parameter. Now lets do the same with the rest of our parameters



- DBRootPassword=$(aws ssm get-parameters --region us-east-1 --names DBrootpassword --with-decryption --query Parameters[0].Value)

- DBRootPassword=`echo $DBRootPassword | sed -e 's/^"//' -e 's/"$//'`



- DBUser=$(aws ssm get-parameters --region us-east-1 --names DBuser  --query Parameters[0].Values)

- DBUser=`echo $DBUser | sed -e 's/^"//' -e 's/"$//'`



- DBName=$(aws ssm get-parameters --region us-east-1 --names DBname --query Parameters[0].Values)

- DBName=`echo $DBName | sed -e 's/^"//' -e 's/"$//'`



- DBEndpoint=$(aws ssm get-parameters --region us-east-1 --names DBendpoint --query Parameters[0].Values)

- DBEndpoint=`echo $DBEndpoint | sed -e 's/^"//' -e 's/"$//'`

Now that we've pulled all the parameters as env variables, we're going to update the server as well as pull the requisites we'll need including the webserver and database.

sudo dnf -y update

sudo dnf install wget php-mysqlnd httpd php-fpm php-mysqli mariadb105-server php-json php php-devel stress -y

Next, set the Webserver and DB to run and start by default so if the instance were to reset, the services don't need to manually restarted

sudo systemctl enable httpd
sudo systemctl enable mariadb
sudo systemctl start httpd
sudo systemctl start mariadb

We also need to set the root password for the DB using the ENV variable we set earlier

sudo mysqladmin -u root password $DBRootPassword

Now install and extract Wordpress. If for whatever reason the download times out, make sure that you chose the pub-A subnet. If you chose a subnet that's not a public subnet, you won't be able to access the public internet and download the file.

sudo wget http://wordpress.org/latest.tar.gz -P /var/www/html
cd /var/www/html
sudo tar -zxvf latest.tar.gz
sudo cp -rvf wordpress/* .
sudo rm -R wordpress
sudo rm latest.tar.gz

Then configure the Wordpress PHP config file with DBName, DBUser, and DBPassword

sudo cp ./wp-config-sample.php ./wp-config.php
sudo sed -i "s/'database_name_here'/'$DBName'/g" wp-config.php
sudo sed -i "s/'username_here'/'$DBUser'/g" wp-config.php
sudo sed -i "s/'password_here'/'$DBPassword'/g" wp-config.php

To be able to access the file we'll have to change permissions for the file as well, which we will do now

sudo usermod -a -G apache ec2-user   
sudo chown -R ec2-user:apache /var/www
sudo chmod 2775 /var/www
sudo find /var/www -type d -exec chmod 2775 {} \;
sudo find /var/www -type f -exec chmod 0664 {} \;

Finally, create the Wordpress user, set its password, and create the DB and configure permissions.

sudo echo "CREATE DATABASE $DBName;" >> /tmp/db.setup
sudo echo "CREATE USER '$DBUser'@'localhost' IDENTIFIED BY '$DBPassword';" >> /tmp/db.setup
sudo echo "GRANT ALL ON $DBName.* TO '$DBUser'@'localhost';" >> /tmp/db.setup
sudo echo "FLUSH PRIVILEGES;" >> /tmp/db.setup
sudo mysql -u root --password=$DBRootPassword < /tmp/db.setup
sudo rm /tmp/db.setup

Lets check to see that everything was installed and configured correctly. Copy the public address of the EC2 instance, its important to open as HTTP not HTTPS or it will not open correctly. If this works, we will proceed with the initial config

in Site Title enter Catagram
in Username enter admin in Password enter 4n1m4l54L1f3
in Your Email enter your email address
Click Install WordPress Click Log In
In Username or Email Address enter admin
in Password enter the previously noted down strong password
Click Log In

Now you should be at the Wordpress home page

Click "posts" in the menu of the left, you should see a "Hello World" post, move that to the trash and apply. Now let's add a new post! Click "add new" and name the site "Best Animals", click "+" under the title, select "gallery" and upload a few animal pictures. Now publish the post.

This is the end of Part 1! This manual process demonstrates the challenges of scaling and maintaining consistency in deployments. In the next part, we’ll leverage launch templates in our Terraform code to streamline deployment, improve scalability, and eliminate manual errors. Stay tuned!

Blog