Alex M. Schapelle
Posted on November 12, 2023
Welcome back gentle reader, I am Silent-Mobius, aka Alex M. Schapelle, your humanoid circuit-board object that will guide you with topics of open-source, software and tools.
In our last article we started to develop a tool that will automate our ISO files configuration.
In this article we'll forge into a tool.
Forging OGUN for editing ISO
Before diving in to the code, I'd suggest, we acknowledge the fact that most of the code today, including mine, are kept in remote repository based mostly but not implicitly on git. You are welcome to visit the Tool repository on gitlab, leave a comment, suggest a feature and clone the tool for your own adventure.
As of now, we have satisfied dependencies, created code snippets and have created working environment. All is left is to combine it all in to one file and add more features. The code below is the updated version of the Tool, which we named OGUN:
#!/usr/bin/env bash
############################ Special Header: Start ################################
# Created by Silent-Mobius, Zero-Pytagoras aka Alex M. Schapelle
# Purpose: Manipulate ISO files
# Version: 01.03.86
# <Manipulate ISO files to configure custom ISO files manually and automoatically inside CI/CD pipelines
#(C) 2023 Silent Mobius aka Alex M. Schapelle
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published
# by the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
set -o errexit
set -o pipefail
set -x
# Global variables
PROJECT="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
CONFIG="$HOME/.config/ogun"
ISO_FOLDER="$PROJECT/iso"
YAML="$PROJECT/config.yaml"
OUT_FOLDER="$PROJECT/iso"
SQUASHFS_FOLDER="$PROJECT/squashfs-root"
DEPENDENCIES=(p7zip-full git genisoimage fakeroot pwgen whois xorriso isolinux binutils squashfs-tools yq)
SEPERATOR="#######################"
############################ Special Header: End ################################
# Messages
_msg_need_root="[!] Escalate permissions with sudo or switch to user root"
_msg_not_support="[!] $ID Distribution is not supported yet "
_msg_reassemble_error="[!] Wrong Value Passed: Please Read Usage with -h flag"
# Value files imports
. /etc/os-release
function main(){
cd $PROJECT
if [[ "$#" == 0 ]];then
usage
exit 0
fi
if [[ $UID -eq 0 ]] || [[ $EUID -eq 0 ]];then
while getopts "chdvDHRSV" tasks
do
case ${tasks[@]} in
D) while getopts "h:i:" OPTS # Disassemble squash and iso files
do
case "${OPTS[@]}" in
h) d_usage; exit 0;;
i) ISO_FILE=$OPTARG ;;
*) d_usage; exit 0;;
esac
done
disassemble_iso $ISO_FILE ${OUT_FOLDER##*/}
disassemble_squashfile $ISO_FOLDER
;;
R) while getopts "h:i:o:" OPTS # Reassemble squash and iso files
do
case "${OPTS[@]}" in
h) r_usage; exit 0;;
i) ISO_FOLDER=$OPTARG;;
o) OS_VERSION=$OPTARG;;
*) r_usage; exit 0;;
esac
done
SQUASH_FILE=$(get_squashfs_file "$SQUASHFS_FOLDER")
reassemble_squashfile $SQUASHFS_FOLDER ${SQUASH_FILE##*/}
sign_md5 $ISO_FOLDER
reassemble_iso $OS_VERSION $ISO_FOLDER
;;
S) while getopts "h:u:v:" OPTS # Setup variables and sed them
do
case "${OPTS[@]}" in
h) s_usage; exit 0;;
v) OS_VERSION=$OPTARG;;
*) s_usage; exit 0;;
esac
done
chroot_squash_folder_with_install_config $SQUASHFS_FOLDER
;;
c) rm -rf "$OUT_FOLDER" "$PROJECT/$SQUASHFS_FOLDER"
;;
d) check_dependencies
;;
h|H)
usage
exit 0;;
v|V) version
exit 0;;
*) usage
exit 0
;;
esac
done
else
deco "$_msg_need_root"
fi
}
function deco(){
IN="$*"
printf "\n$SEPERATOR\n %s\n$SEPERATOR\n" "$IN"
}
function version(){
VERSION=$(grep -m1 '# Version: ' $0 | awk '{print $3}')
deco "[>] Version: $VERSION"
}
function d_usage(){
deco "
[>>] -h : Disassembly function descriptionss
[>>] -i : ISO file which to disassemble
[>>] Example: $0 -D -i ubuntu-22.04.3-live-server-amd64.iso
"
}
function r_usage(){
deco "
[>>] -h : Reassemble function description
[>>] -i : Exported ISO folder path. E.g. iso folder
[>>] Example: $0 -R -i iso
"ca
}
function s_usage(){
deco "
[>>] -h : Setup function descriptions
[>>] -v : OS version --> major version number : 20 and 22 in case of ubuntu
[>>] Example: $0 -S -v 22
"
}
function usage(){
clear
deco "[!] Incorect Usage:
[>] -c: Clean up the working
[>] -d: Dependency check
[>] -h|H: This help message
[>] -D: Disassemble ISO and SQuashfs file
[>>] -h : this help function
[>>] -i : ISO file which to disassemble
[>>] Example: $0 -D -i ubuntu-22.04.3-live-server-amd64.iso
[>] -R: Reassemble ISO and SQuashfs file
[>>] -h : Reassemble function description
[>>] -i : Exported ISO folder path. E.g. iso folder
[>>] Example: $0 -R -i iso
[>] -S: Setup Configuration for Configuration
[>>] -h : this help function
[>>] -v : OS version 12, 20 and 22 for major definitions
[>>] Example: $0 -S -v 22
"
}
function disassemble_iso(){
local IN=$1 # full path is required ...
local OUT=$2
7z x -y $IN -o$OUT
}
function get_squashfs_file(){
local IN=$1
local min=100
local tmp_squash_file=''
SQUASH_FILES=($(find $IN -name '*.squashfs'))
for squash_file in "${SQUASH_FILES[@]}"
do
if [[ ${squash_file##*/} == 'filesystem.squashfs' ]];then
echo $squash_file
break
fi
if [[ ${#squash_file} -lt $min ]] || [[ ${#squash_file} -le $min ]];then
min=${#squash_file}
tmp_squash_file=${squash_file}
fi
done
echo "$tmp_squash_file"
}
function disassemble_squashfile(){
local PATH_TO_SQUASH_FOLDER=$1
local SQUASH_FILE_PATH=$(get_squashfs_file "$PATH_TO_SQUASH_FOLDER")
mv "$SQUASH_FILE_PATH" "./${SQUASH_FILE_PATH##*/}"
unsquashfs "./${SQUASH_FILE_PATH##*/}"
rm -rf $PROJECT/${SQUASH_FILE_PATH##*/}
}
function reassemble_squashfile(){
local squash_folder=$1
local squash_file=$2
mksquashfs $squash_folder $squash_file -comp xz -b 1M -noappend
mv "$squash_file" "$OUT_FOLDER/isolinux/"
}
function reassemble_iso(){
local system_version=$1
local iso_folder=$2
if [[ ! ${system_version} ]] || [[ ${#system_version} -gt 2 ]];then
deco $_msg_reassemble_error
exit 1
else
if [[ ${system_version} -eq 22 ]];then
xorriso -as mkisofs -r -V 'Unattended Custom Install' --grub2-mbr ./BOOT/1-Boot-NoEmul.img -partition_offset 16 --mbr-force-bootable \
-append_partition 2 28732ac11ff8d211ba4b00a0c93ec93b ./BOOT/2-Boot-NoEmul.img -appended_part_as_gpt \
-iso_mbr_part_type a2a0d0ebe5b9334487c068b6b72699c7 -c '/boot.catalog' -b '/boot/grub/i386-pc/eltorito.img' \
-no-emul-boot -boot-load-size 4 -boot-info-table --grub2-boot-info -eltorito-alt-boot -e '--interval:appended_partition_2:::' \
-no-emul-boot -o "iso-deb${system_version}.iso" $iso_folder
else
xorriso -as mkisofs -r -V "Unattended Custom Install" -J -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot\
-boot-load-size 4 -isohybrid-mbr /usr/lib/ISOLINUX/isohdpfx.bin -boot-info-table -input-charset utf-8 -eltorito-alt-boot\
-e boot/grub/efi.img -no-emul-boot -isohybrid-gpt-basdat -o "iso-deb${system_version}.iso" $iso_folder
fi
fi
}
function sign_md5(){
local iso_folder=$1
md5sum "$iso_folder/.disk/info" > "$iso_folder/md5sum.txt"
sed -i "s|$iso_folder/|./|g" "$iso_folder/md5sum.txt"
}
function chroot_squash_folder_with_install_config(){
local CHROOT=$1
# readarray -t repos < <(read_yaml_config '.repositories|values[]'|tr -d '"')
readarray -t packages < <(read_yaml_config .install.apt_package_list[]|tr -d '"')
# TODO: implement cert install correctly
curl -sL https://www.virtualbox.org/download/oracle_vbox_2016.asc -o "$CHROOT/usr/share/keyrings/oracle-virtualbox-2016.gpg"
curl -sL https://gitlab.com/paulcarroty/vscodium-deb-rpm-repo/raw/master/pub.gpg -o "$CHROOT/usr/share/keyrings/vscodium-archive-keyring.gpg"
curl -sL https://apt.releases.hashicorp.com/gpg -o "$CHROOT/usr/share/keyrings/hashicorp-archive-keyring.gpg"
# TODO: repositories are not still added in key valye manner:
# TODO: implementation: key: name of repo | value :string of repo
chroot $CHROOT /bin/bash -c "
echo 'nameserver 8.8.8.8' > /etc/resolv.conf
apt-get update
DEBIAN_FRONTEND=noninteractive apt-get install -y ${packages[@]}
apt-cache clean
rm -rf /var/cache/apt/archives
rm -rf /var/lib/apt/lists
echo '' > /etc/resolv.conf
history -c
"
}
#################################### utils ################################################3
function is_root(){
if [[ $UID -eq 0 ]] || [[ $EUID -eq 0 ]]; then
return 0
fi
return 1
}
function check_dependencies(){
if [[ ! -e "${CONFIG}/.dependencies_exists" ]];then
if [[ ! "$CONFIG" ]];then
mkdir -p "$CONFIG"
fi
if [[ $ID == 'debian' ]] || [[ $ID == 'ubuntu' ]];then
INSTALLER='apt-get'
else
deco "$_msg_not_support"
exit 1
fi
for package in "${DEPENDENCIES[@]}"
do
if ! which "$package" > /dev/null 2>&1;then
$INSTALLER -y install "$package"
fi
done
fi
}
function read_yaml_config(){
local config_table=$1
local array_list=""
for element in $(yq -r < $YAML $config_table)
do
array_list=("${array_list[@]}" "$element")
done
echo "${array_list[@]}"
}
function set_repositories(){
repo_name=$1
repo_address=$2
touch "$CHROOT/etc/apt/sources.list.d/$repo_name.list"
echo "$repo_address"| tee "$CHROOT/etc/apt/sources.list.d/$repo_name"
}
function is_exists(){
local _file=$1
if [[ -e $_file ]];then
return 0
fi
return 1
}
########
# Main - _- _- _- _- _- _- _- Do Not Remove - _- _- _- _- _- _- _- _- _- _- _- _- _
########
main "$@"
[+] Note: In order to use OGUN, configuration file is required, thus DO NOT try to RUN the script YET.
[+] Note: Configuration is suppose to be namedconfig.yaml, and for now it supports, only package install from repositories, and needs to be structured as associate dictinary including array of packages installed.
[$] Explanation: I love using environment variables in conjunction with positional arguments and optional arguments, so as we continue to run commands, we'll also write bash functions in most general way possible and combine them with types of variables mentioned before. All and all it will allow us to convert bash functions in to shell based tool for manipulating ISO type files.
Although short article, it has a complex code, and lots of fixes to do, which you may follow on gitlab and You may continue reading about using ogun in CI/CD with Jenkins in our next article
Till then, remember: Do Try To Have Some Fun
💖 💪 🙅 🚩
Alex M. Schapelle
Posted on November 12, 2023
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.