Day 016 - 100DaysAWSIaCDevopsChallenge

In my previous post, Step-by-Step Guide to Setting Up and Deploying SonarQube on an AWS EC2 Instance Using AWS Cloud Development Kit (CDK)[↗], I demonstrated how to deploy a SonarQube Server on an EC2 instance. As the database were self-hosted in EC2 Instance, today, I will show you how to extend the database by integrating it with an external service like Amazon RDS or Aurora, rather than keeping it self-hosted within the server.

Why did I need to extend the SonarQube Database to a RDS PostgreSQL Instance?

Extending SonarQube's database to an external service like Amazon RDS (Relational Database Service) or Aurora (Amazon Severless Database managed) instead of keeping it inside the self-hosted SonarQube sever offers several advantages:

1. Scalability: Managed database like RDS and Aurora can handle inscreased workloads, provide more robust storage and offer auto-scaling feature by ensuring best permformance as your SonarQube instance grows.

2. Reliability: Provide high availability through automatic backups, replication and failover mechanisms. They reduce the risk of data loss and downtime in case of hardware failure.

3. Security: Offer built-in security features, including encryption at rest and in transit, network isolation through VPC and private subnet making your data more secure than self-managed (self-hosted) databases.

4. Maintainance: AWS handles regular maintenance tasks, such as backups, patching, and updates, reducing the burden on your infrastructure team.

Update the previous infrastructure to setting up a RDS PostgreSQL Instance for SonarQube

The infrastructure consists of the following components:

• VPC and two (02) Subnets(private and public subnet) to manage network traffic within the server and database instance.
• Internet Gatewayto allow internet communication between our secured cloud environment and external networks.
• Security group to control access to the server and database based on IP addresses and ports connections.
• EC2 Instance to host the SonarQube server (including the Web Server, ElasticSearch, and Postgres Database).
• Key Pair the private key that allows external hosts to connect securely to the instance.
• Secret Manager to securely store the database user password.
• IAM Role to allow EC2 Instance to connect to Secret Manager and retrieve database password
• Elastic IP (EIP) - The static IP to attach to the EC2 Instance. It allows to maintain a fixed IP address aven if the instance is stopped of restarted.
• Nat Gateway to allow instance (here database instance) in a private subnet to connect to the internet or other AWS Service without exposing then to internet traffic.
• RDS PostgreSQL the database instance to host the sonarqube data.

Create the database Construct

interface DatabaseProps extends CustomProps {
  vpc: IVpc,
  sg?: ISecurityGroup // the security group for SonarQube Server Instance
}
export class Database extends Construct {
  private readonly _dbInstanceArn: string
  private readonly _passwordSecretArn: string
  private readonly _instanceEndpointUrl: string
  constructor(scope: Construct, id: string, props: DatabaseProps) {
    super(scope, id)
    const secret: ISecret = new aws_secretsmanager.Secret(this, 'SecretResource', {
      secretName: 'database-instance-secret',
      generateSecretString: {
        secretStringTemplate: JSON.stringify({ username: props.databaseUsername }),
        generateStringKey: 'password',
        excludeCharacters: '/@"`()[]\'',
        includeSpace: false
      }
    })
    const securityGroup = new SecurityGroup(this, 'DatabaseSecurityGroupeResource', {
      securityGroupName: 'sq-database-sg',
      disableInlineRules: false,
      allowAllOutbound: true,
      vpc: props.vpc,
      description: 'Database instance security group'
    })
    securityGroup.addIngressRule(
      Peer.securityGroupId(props.sg!.securityGroupId),
      Port.tcp(props.databasePort ?? 5432), 'Allow EC2 to connect to the database instance')
    const db = new rds.DatabaseInstance(this, 'DatabaseInstanceResource', {
      engine: rds.DatabaseInstanceEngine.postgres({
        version: rds.PostgresEngineVersion.VER_12_16
      }),
      vpc: props.vpc,
      networkType: rds.NetworkType.IPV4,
      vpcSubnets: props.vpc.selectSubnets({ subnetType: SubnetType.PRIVATE_WITH_EGRESS }),
      instanceType: InstanceType.of(InstanceClass.BURSTABLE3, InstanceSize.MICRO),
      publiclyAccessible: false,
      storageType: StorageType.GP3,
      credentials: rds.Credentials.fromSecret(secret, props.databaseUsername),
      allocatedStorage: 20,
      databaseName: props.databaseName,
      instanceIdentifier: 'sonarqube-db-1',
      port: props.databasePort ?? 5432,
      securityGroups: [
        securityGroup
      ]
    })
    this._dbInstanceArn = db.instanceArn
    this._passwordSecretArn = secret.secretArn
    this._instanceEndpointUrl = db.instanceEndpoint.socketAddress
  }
  get dbInstanceArn(): string {
    return this._dbInstanceArn
  }
  get passwordSecretArn() {
    return this._passwordSecretArn
  }
  get instanceEndpointUrl(): string {
    return this._instanceEndpointUrl
  }
}

The important thing to know here is the snippet code:

securityGroup.addIngressRule(
    Peer.securityGroupId(props.sg!.securityGroupId),
    Port.tcp(props.databasePort ?? 5432), 'Allow EC2 to connect to the database instance')

This code allows communication between the database and the EC2 instance server through the specified port.

Update the Stack

This is the update for the stack created in the previous article [↗].

export class SonarServerInfrastructureStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props: CustomProps) {
    super(scope, id, props)
    ....
    const vpc = new ec2.Vpc(this, 'PublicVpcResource', {
      ...
      maxAzs: 2,
      subnetConfiguration: [{
        subnetType: ec2.SubnetType.PUBLIC,
        name: 'sq-public-subnet-webserver',
        mapPublicIpOnLaunch: true
      }, {
        subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS,
        name: 'sq-private-subnet-database'
      }]
    })

    sonarQubeSG.addEgressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(props.databasePort ?? 5432), 'Allow outbound traffic to database instance')

    // set up the database instance
    const db = new Database(this, 'DatabaseResource', {
      ...props,
      vpc: vpc,
      sg: sonarQubeSG
    })

    const sonarQubeServer = new ec2.Instance(this, 'SonarQubeServerInstanceResource', {
      ...
      role: new iam.Role(this, 'Ec2RoleResource', {
        roleName: 'ec2-role-and-permission',
        assumedBy: new iam.ServicePrincipal('ec2.amazonaws.com'),
        path: '/',
        inlinePolicies: {
          secretsManager: new iam.PolicyDocument({
            assignSids: true,
            statements: [
              new iam.PolicyStatement({
                effect: iam.Effect.ALLOW,
                actions: [
                  'secretsManager:GetSecretValue',
                  'secretsManager:DescribeSecret'
                ],
                resources: [db.passwordSecretArn],
                conditions: {}
              })
            ]
          })
        }
      })
    })

    const userDataEncoded = ec2.UserData.forLinux()
    userDataEncoded.addCommands(fs.readFileSync('./assets/sonarqube-server.sh', 'utf-8'))
    sonarQubeServer.addUserData(userDataEncoded.render()
      .replace('{{DB_NAME}}', props.databaseName || 'sonarqube')
      .replace('{{DB_USER}}', props.databaseUsername || 'sonar')
      .replace('{{SECRET_ARN}}', db.passwordSecretArn)
      .replace('{{DB_SOCKET_ADDRESS}}', db.instanceEndpointUrl))
  }
}

• PRIVATE_WITH_EGRESS - In the vpc configuration, we had added a new entry of subnet of type PRIVATE_WITH_EGRESS, meaning that it can access the internet but is not exposed to inbound internet traffic.
• Egress rule - Allows the SonarQube server to send traffic to the database instance on the specified port (default 5432 for PostgreSQL)

• New User data scripts As we no longer need the self-hosted database, we have to replace the database connection properties with the new ones:
  

  ## Update package index
  sudo apt update
  
  ## Install OpenJDK 17
  sudo apt install -y openjdk-17-jdk zip unzip
  ## Set up the JAVA_HOME environment variable
  echo "export JAVA_HOME=$(dirname $(dirname $(readlink -f $(which java))))" | sudo tee /etc/profile.d/jdk.sh
  source /etc/profile.d/jdk.sh
  
  ## download sonarqube server - community version
  wget https://binaries.sonarsource.com/Distribution/sonarqube/sonarqube-9.9.6.92038.zip
  
  ## unzip content
  unzip sonarqube-*.zip
  sudo mv sonarqube-9.9.6.92038 sonarqube
  sudo mv sonarqube /opt/
  
  ## Create sonarqube user and its group
  sudo groupadd sonarqube
  sudo useradd -d /opt/sonarqube -g sonarqube sonarqube
  sudo chown -R sonarqube:sonarqube /opt/sonarqube
  
  ## Insert user runner after `APP_NAME="SonarQube"` line
  sudo sed -i 's/APP_NAME="SonarQube"/APP_NAME="SonarQube"\n\nRUN_AS_USER=sonarqube/' /opt/sonarqube/bin/linux-x86-64/sonar.sh
  
  ## create sonarqube launcher service
  sudo tee /etc/systemd/system/sonarqube.service <<EOF
  [Unit]
  Description=SonarQube service
  After=syslog.target network.target
  [Service]
  Type=forking
  User=sonarqube
  Group=sonarqube
  PermissionsStartOnly=true
  ExecStart=/opt/sonarqube/bin/linux-x86-64/sonar.sh start
  ExecStop=/opt/sonarqube/bin/linux-x86-64/sonar.sh stop
  StandardOutput=journal
  LimitNOFILE=131072
  LimitNPROC=8192
  TimeoutStartSec=5
  Restart=always
  SuccessExitStatus=143
  
  [Install]
  WantedBy=multi-user.target
  EOF
  
  ## Installing AWS-CLI
  sudo snap install aws-cli --classic
  
  sudo tee /opt/sonarqube/conf/sonar.properties <<EOF
  sonar.web.port=9000
  sonar.jdbc.username={{DB_USER}}
  sonar.jdbc.password=$(aws secretsmanager get-secret-value --secret-id {{SECRET_ARN}} | jq --raw-output '.SecretString' | jq -r .password)
  sonar.jdbc.url=jdbc:postgresql://{{DB_SOCKET_ADDRESS}}/{{DB_NAME}}
  EOF
  
  ## Start SonarQube Server
  sudo systemctl enable sonarqube.service
  sudo systemctl start sonarqube.service
  

Deploy the stack

const app = new cdk.App()
new SonarServerInfrastructureStack(app, 'Day016Stack', {
  cidr: '10.0.0.1/16',
  databaseName: 'sonarqubedb',
  databaseUsername: 'sonar',
  databasePort: 5000,
  env: {
    region: process.env.CDK_DEFAULT_REGION,
    account: process.env.CDK_DEFAULT_ACCOUNT
  }
})
app.synth()

Open the terminal anywhere and run the following commande:

git clone https://github.com/nivekalara237/100DaysTerraformAWSDevops.git
cd 100DaysTerraformAWSDevops/day_016
cdk deploy --profile cdk-user Day016Stack

After the deployment, open the browser and type this url http://PUBLIC_IP_or_DNS:9000

__

🥳✨
We have reached the end of the article.
Thank you so much 🙂

Your can find the full source code on GitHub Repo↗