Automated Backups Made Easy: & Lessons Learned From This Project

Table of Contents:

Introduction
Project Overview
Discoveries and Resolutions
- Use Full Paths in Crontabs
- Specify Full Paths for Rsync and SSH Key
Conclusion

Introduction:

In the ever-evolving world of technology, ensuring the safety of our data is paramount. Recently, I completed a project that automates backup tasks, aiming to simplify the process and enhance data security. While the project proved successful, I encountered a stumbling block that led me on a troubleshooting journey. In this blog post, I'll share the project overview and the valuable lessons learned from resolving a crucial issue.

Project Overview:

The project (GitHub link: Auto-Backup Project) automates the addition of backup tasks to the crontab for daily, weekly, and monthly backups. All compressions (.tar.gz) are stored in a designated backup directory, and SSH key authentication is required for remote hosts. The script intelligently manages backup files, retaining only the most recent ones to optimize storage usage.

Discoveries and Resolutions:

While developing the project, a significant challenge surfaced when incorporating the rsync command into the script for scheduled execution via crontab. Here are the two major discoveries that solved the issue:

Use Full Paths in Crontabs:

Crontab entries should include the full path to scripts. Correct example:
```
 * * * * * /usr/local/bin/backup.sh
```

Specify Full Paths for Rsync and SSH Key

In the script, provide the full path to the rsync program and explicitly specify the SSH key for the remote system.
```
 /usr/bin/rsync -a -e "ssh -i /home/user/.ssh/vps-1-private-key" /home/user/backups user@vps-1.com
```
To troubleshoot, add verbose logging with the -vv option to the rsync in the script and redirect the crontab output to a file for monitoring.
```
 * * * * * /usr/local/bin/backup.sh >> /home/user/.backup.cron.log 
```

Conclusion:

Developing the Auto-Backup Project was a rewarding experience, and overcoming the challenge highlighted the importance of attention to detail. By sharing these discoveries, I hope to assist fellow developers facing similar issues. Remember, the devil is in the details, and providing full paths can make all the difference in the world of scheduled tasks. Explore the GitHub repository for more details and project documentation.

Check out a related problem posted on StackOverFlow

Also, connect with me on LinkedIn

Happy hacking!

Blog