Practical Usage of Shell Scripting in Devops/Cloud Engineer Role

Shell scripting is an essential tool for DevOps and Cloud Engineers, offering automation, efficiency, and consistency in managing complex environments. Here are some practical use cases of shell scripting in these roles, along with example scripts:

1. Automating Infrastructure Deployment

Use Case: Automate the deployment of infrastructure resources such as EC2 instances, RDS databases, or Kubernetes clusters using CLI tools.

Example Script: Deploying an EC2 instance using AWS CLI.

#!/bin/bash

# Variables
INSTANCE_TYPE="t2.micro"
KEY_NAME="my-key-pair"
AMI_ID="ami-0abcdef1234567890"
SECURITY_GROUP="sg-0abc1234"
SUBNET_ID="subnet-0abc1234"
REGION="us-west-2"

# Launch EC2 Instance
aws ec2 run-instances \
    --instance-type $INSTANCE_TYPE \
    --key-name $KEY_NAME \
    --image-id $AMI_ID \
    --security-group-ids $SECURITY_GROUP \
    --subnet-id $SUBNET_ID \
    --region $REGION \
    --count 1 \
    --output json

echo "EC2 Instance launched successfully."

2. Continuous Integration and Deployment (CI/CD)

Use Case: Automate build, test, and deployment pipelines.

Example Script: Build and deploy a Docker container to AWS ECS.

#!/bin/bash

# Variables
ECR_REPOSITORY="my-ecr-repo"
IMAGE_TAG=$(date +%Y%m%d%H%M%S)
CLUSTER_NAME="my-cluster"
SERVICE_NAME="my-service"

# Authenticate Docker with ECR
aws ecr get-login-password --region us-west-2 | docker login --username AWS --password-stdin $ECR_REPOSITORY

# Build Docker Image
docker build -t $ECR_REPOSITORY:$IMAGE_TAG .

# Push Docker Image
docker push $ECR_REPOSITORY:$IMAGE_TAG

# Update ECS Service
aws ecs update-service --cluster $CLUSTER_NAME --service $SERVICE_NAME --force-new-deployment

echo "Deployment completed successfully."

3. System Monitoring and Alerts

Use Case: Monitor system health and send alerts if certain thresholds are breached.

Example Script: Check disk usage and send an alert if usage exceeds 80%.

#!/bin/bash

# Variables
THRESHOLD=80
EMAIL="admin@example.com"
HOSTNAME=$(hostname)

# Check Disk Usage
DISK_USAGE=$(df / | grep / | awk '{ print $5 }' | sed 's/%//g')

if [ "$DISK_USAGE" -gt "$THRESHOLD" ]; then
    echo "Disk usage on $HOSTNAME is above threshold: ${DISK_USAGE}%" | mail -s "Disk Usage Alert" $EMAIL
    echo "Alert sent to $EMAIL."
else
    echo "Disk usage on $HOSTNAME is within limits: ${DISK_USAGE}%."
fi

4. Log Management

Use Case: Aggregate and analyze logs from multiple servers.

Example Script: Collect logs from multiple servers and compress them for archival.

#!/bin/bash

# Variables
LOG_DIR="/var/log/myapp"
ARCHIVE_DIR="/var/archives/logs"
TIMESTAMP=$(date +%Y%m%d%H%M%S)
ARCHIVE_FILE="$ARCHIVE_DIR/logs-$TIMESTAMP.tar.gz"

# Create Archive Directory if Not Exists
mkdir -p $ARCHIVE_DIR

# Collect Logs
tar -czf $ARCHIVE_FILE $LOG_DIR

# Remove Old Archives (older than 30 days)
find $ARCHIVE_DIR -type f -mtime +30 -exec rm {} \;

echo "Logs collected and archived to $ARCHIVE_FILE."

5. Backup and Restore

Use Case: Automate backup processes for databases or file systems.

Example Script: Backup a MySQL database and compress the backup file.

#!/bin/bash

# Variables
DB_USER="root"
DB_PASSWORD="password"
DB_NAME="mydatabase"
BACKUP_DIR="/backups"
TIMESTAMP=$(date +%Y%m%d%H%M%S)
BACKUP_FILE="$BACKUP_DIR/db-backup-$TIMESTAMP.sql.gz"

# Create Backup
mysqldump -u $DB_USER -p$DB_PASSWORD $DB_NAME | gzip > $BACKUP_FILE

# Remove Old Backups (older than 7 days)
find $BACKUP_DIR -type f -mtime +7 -exec rm {} \;

echo "Database backup completed: $BACKUP_FILE."

6. Environment Configuration

Use Case: Set up environment variables and configuration files.

Example Script: Set up environment variables for a web application.

#!/bin/bash

# Variables
ENV_FILE="/etc/myapp/.env"

# Create Environment File
echo "DB_HOST=localhost" > $ENV_FILE
echo "DB_USER=root" >> $ENV_FILE
echo "DB_PASSWORD=password" >> $ENV_FILE
echo "PORT=8080" >> $ENV_FILE

echo "Environment configuration completed."

Summary

Shell scripting in DevOps and cloud engineering helps automate routine tasks, streamline workflows, and ensure consistency across environments. By writing scripts for infrastructure deployment, CI/CD pipelines, system monitoring, log management, backup, and environment configuration, you can improve efficiency and reduce manual effort in managing and operating cloud and on-premises systems.

Blog