GBase 数据库 | GBase 8a MPP Cluster Automatic Log Cleanup Implementation

Efficient log management is essential for the stable operation of GBase 8a MPP Clusters. This article demonstrates how to set up an automated script to regularly clear system logs, helping to optimize storage use and streamline maintenance tasks in the GBase database (GBase数据库).

Automatic Log Cleanup

During the maintenance and operation of GBase 8a, it is often necessary to regularly clean up system logs. This can be achieved by configuring a script in crontab as described below.

Crontab Configuration

An example configuration to run the script on the 1st day of each month is shown below.

1） Grant execute permission to the cleanlogs.sh script for the gbase user:

   chmod +x /opt/gbasetools/cleanlogs.sh

2） Edit the crontab for the gbase user:

   crontab -e

3） Sample crontab configuration:

   0 0 1 * * sh /opt/gbasetools/cleanlogs.sh

4） Check the crontab configuration:

   crontab -l

Script Explanation

Parameter: ceiling defines the maximum log file size, which can be adjusted as needed.
Execution: The script should be executed by the gbase user.
Cleanup Scope:
- Cluster-level express logs
- Node-level express logs
- Cluster-level system logs
- Node-level system logs
- gcware logs
- loader logs
Loader Log Cleanup Rule: Logs from directories dated before the current day are deleted. To adjust, modify the -mtime +N parameter in the script.

Script Content

########################################################################################
# script name : cleanlogs.sh
# author: zhaoqinggang
# create time: 2024-09-25
# specification: Cluster Log Cleanup Script
# precondition: Deploy this script in the `crontab` of each node under the `gbase` user
# run user: gbase
########################################################################################

ceiling=10000000000   # 10GB

gc_log_home="$GCLUSTER_BASE/log/gcluster"
gn_log_home="$GBASE_BASE/log/gbase"
loader_log_home="$GCLUSTER_BASE/log/gcluster/loader_logs"
gcware_home="$GCWARE_BASE/log/"

c_gc_express()
{
   doc="$gc_log_home/express.log"
   fsize=`stat -c %s $doc`
   if [ $fsize -gt $ceiling ]; then
      echo "" > $doc
   fi
}

c_gn_express()
{
   doc="$gn_log_home/express.log"
   fsize=`stat -c %s $doc`
   if [ $fsize -gt $ceiling ]; then
      echo "" > $doc
   fi
}

c_gc_system()
{
   doc="$gc_log_home/system.log"
   fsize=`stat -c %s $doc`
   if [ $fsize -gt $ceiling ]; then
      echo "" > $doc
   fi
}

c_gn_system()
{
   doc="$gn_log_home/system.log"
   fsize=`stat -c %s $doc`
   if [ $fsize -gt $ceiling ]; then
      echo "" > $doc
   fi
}

c_gcware()
{
   doc="$gcware_home/gcware.log"
   fsize=`stat -c %s $doc`
   if [ $fsize -gt $ceiling ]; then
      echo "" > $doc
   fi
}

c_gc_loader_log()
{
   # Delete directories with logs prior to the current day
   cd $loader_log_home && find . -type d -mtime +0 | grep -v "^.$" | xargs rm -rf
}

main()
{
   c_gc_express
   c_gn_express
   c_gc_system
   c_gn_system
   c_gcware
   c_gc_loader_log
}

main

By setting up this automated cleanup process, you can keep log sizes under control and ensure smoother operations for your GBase database (GBase数据库).

Blog

GBase 数据库 | GBase 8a MPP Cluster Automatic Log Cleanup Implementation

Cong Li

Automatic Log Cleanup

Crontab Configuration

Script Explanation

Script Content

Join Our Newsletter. No Spam, Only the good stuff.

Related