Getting information

Return your user name

whoami

Return your user and group id

id

Return operating system name, username, and other info

uname -a

Display reference manual for a command

man top

Get help on a command

curl --help

Return the current date and time

date

Monitoring performance and status

List selection of or all running processes and their PIDs

ps
ps -e

Display resource usage

top

List mounted file systems and usage

df

Working with files

Copy a file

cp file.txt new_path/new_name.txt

Change file name or path

mv this_file.txt that_path/that_file.txt

Remove a file verbosely

rm this_old_file.txt -v

Create an empty file, or update existing file's timestamp

touch a_new_file.txt

Change/modify file permissions to 'execute' for all users

chmod +x my_script.sh

Get count of lines, words, or characters in file

wc -l table_of_data.csv
wc -w my_essay.txt
wc -m some_document.txt

Return lines matching a pattern from files matching a filename pattern - case insensitive and whole words only

grep -iw hello \*.txt

Return file names with lines matching the pattern 'hello' from files matching a filename pattern

grep -l hello \*.txt

Navigating and working with directories

List files and directories by date, newest last

ls -lrt

Find files in directory tree with suffix 'sh'

find -name '\*.sh'

Return present working directory

pwd

Make a new directory

mkdir new_folder

Change the current directory: up one level, home, or some other path

cd ../
cd ~ or cd
cd another_directory

remove directory, verbosely

Printing file and string contents

Print file contents

cat my_shell_script.sh

Print file contents page-by-page

more ReadMe.txt

Print first N lines of file

head -10 data_table.csv

Print last N lines of file

tail -10 data_table.csv

Print string or variable value

echo "I am not a robot"
echo "I am $USERNAME"

Compression and archiving

Archive a set of files

tar -cvf my_archive.tar.gz file1 file2 file3

Compress a set of files

zip my_zipped_files.zip file1 file2
zip my_zipped_folders.zip directory1 directory2

Extract files from a compressed zip archive

unzip my_zipped_file.zip
unzip my_zipped_file.zip -d extract_to_this_direcory

Performing network operations

Print hostname

hostname

Send packets to URL and print response

ping www.google.com

Display or configure system network interfaces

ifconfig
ip

Display contents of file at a URL

curl <url>

Download file from a URL

wget <url>

Bash shebang

!/bin/bash

Pipes and Filters

Chain filter commands using the pipe operator

ls | sort -r

Pipe the output of manual page for ls to head to display the first 20 lines

man ls | head -20

Shell and Environment Variables

List all shell variables

set

Define a shell variable called my_planet and assign value Earth to it

my_planet=Earth

Display shell variable

echo $my_planet

List all environment variables

env

Environment vars: define/extend variable scope to child processes

export my_planet
export my_galaxy='Milky Way'

Metacharacters

Comments

# The shell will not respond to this message

Command separator

echo 'here are some files and folders'; ls

File name expansion wildcard

ls *.json

Single character wildcard

ls file_2021-06-??.json

Quoting

Single quotes - interpret literally

echo 'My home directory can be accessed by entering: echo $HOME'

Double quotes - interpret literally, but evaluate metacharacters

echo "My home directory is $HOME"

Backslash - escape metacharacter interpretation

echo "This dollar sign should render: \$"

I/O Redirection

Redirect output to file

echo 'Write this text to file x' > x

Append output to file

echo 'Add this line to file x' >> x

Redirect standard error to file

bad_command_1 2> error.log

Append standard error to file

bad_command_2 2>> error.log

Redirect file contents to standard input

$ tr “[a-z]” “[A-Z]” < a_text_file.txt

The input redirection above is equivalent to

$cat a_text_file.txt | tr “[a-z]” “[A-Z]”

Command Substitution

Capture output of a command and echo its value

THE_PRESENT=$(date)
echo "There is no time like $THE_PRESENT"

Command line arguments

./My_Bash_Script.sh arg1 arg2 arg3

Batch vs. concurrent modes
Run commands sequentially

start=$(date); ./MyBigScript.sh ; end=$(date)

Run commands in parallel

./ETL_chunk_one_on_these_nodes.sh & ./ETL_chunk_two_on_those_nodes.sh

Scheduling jobs with Cron

Open crontab editor

crontab -e

Job scheduling syntax

m h dom mon dow command

minute, hour, day of month, month, day of week
'*' means any

Append the date/time to file every Sunday at 6:15 pm

15 18 * * 0 date >> sundays.txt

Run a shell script on the first minute of the first day of each month

1 0 1 * * ./My_Shell_Script.sh

Back up your home directory every Monday at 3 am

0 3 * * 1 tar -cvf my_backup_path\my_archive.tar.gz $HOME\

Deploy your cron job

Close the crontab editor and save the file

List all cron jobs

crontab -l