A Senior Dev's List of Key Unix Utilities, Explained by a Junior Dev
dianakw8591
Posted on August 7, 2020
If you are a developer working on a Mac, chances are you've used a Unix utility or two. At least to cd
around to different directories or touch
a new file. But chances are you haven't used the vast majority of Unix programs you have on your machine - I have 1325 on mine! Luckily, there's no need to sit down and memorize them all. What is important is to recognize the power of the Unix command line, and to know a few commands that will make your life in the terminal much easier. Below is a list of those commands. But first, a couple notes about using Unix commands in general.
General Rules of Unix Utilities
- Commands are lowercase
- Options modify the command. They are often single letters added in short form with a single dash
-
. Some have long form options added with a double dash--
. - Options can be combined. For example, adding the
-a
and-l
options tols
can be written asls -a -l
orls -al
- Commands can take arguments, often filenames
- Options come before filenames
- There must be spaces between commands, options, and filenames
Unix Utilities to Know by Heart
ls
Lists all your files in the current directory. There are a few important option flags to know as well.
-
-a
will also show all the hidden files in a directory (those that start with a.
). -
-l
will list the files in long format, showing permissions, size, and modified date -
-lh
lists in long format with readable file size (eg 1K, 23M) -
-s
lists the allocated size of each file in blocks
cp source destination
Copies the source to the destination. Source can be one or more files or directories. Destination is the name of the directory to copy to or the name of the new file. When copying directories, the -r
(or -R
) flag must be used. This copies recursively, and will ensure that the contents of a folder is also copied.
cd
Change directory. cd
with no arguments will bring you to your home directory, which is represented by ~
. You can also pass a pathname or make smaller "steps" with cd ..
, which will move to your parent directory. cd
will take relative or absolute pathnames, which can be built with ~
(home directory), ..
(parent directory), .
(current directory) and /
within directories. Note that if you are in a directory with a subfolder test
, cd test
is equivalent to cd ./test
and both will move you into the test folder.
mv source destination
Moves (or renames) source to destination. If the destination is an existing directory, source(s) will be moved there.
rm source
Removes files or directories. Similarly to cp
, to remove directories the -r
or -R
flag must be used which will recursively removed the contents of the directory.
mkdir directory
Create a new directory. The -p
flag is used to create parent directories if they don't already exist. For example, if I am in a directory with a subdirectory test
and want to create a director example
within test
, I could run mkdir -p test/example
. Since test already exists, a new folder example
would be created in test
. However, if test
didn't exist, the command would simply create test
and then create example
within test. Either way there are no errors.
grep patterns file
Searches for patterns in each file, and prints each line that matches. This is a really useful command for finding other commands you've used if they are in your logs. The -r
flag reads files in subdirectories recursively. -n
adds to the output the line number in the file where the match occurred.
cat filename
Used to view the contents of a file. With an argument of a filename, cat
will simply print the content of that file. Other options can be added to view only parts of the file or to write contents to a new file.
echo
Displays a line of text. Most often used in scripts and other commands where you need to insert text.
head filename
Output the first part of the file. By default, head
prints the first 10 lines. Useful when you only need to see the beginning of a file with lots of content.
tail filename
Output the last part of the file. Like head
but for the end of the file.
less filename
Opens a file in the terminal and allows forward and backward navigation within the file. In this mode, patterns can be searched for with /pattern
and using n
and N
to move backwards and forwards between matches. There are a bunch more navigation options to use as well. q
will exit navigation.
Unix Utilities to Know About
find
Searches for files based on some user specified criteria, and returns a list of the found files. The search criteria could be pattern matching, a file type, or file name to name a few. For example, find . -name 'test*'
would search the current directory for filenames that started with 'test', with *
being a placeholder for anything that might follow.
curl
A tool to transfer data to or from a server, using any of its supported protocols (FTP or HTTP for example). There are whole bunch of options for using curl
but know that it exists so you can search for the specifics when you need it! wget
is a similar command with a couple differences, mainly its ability to download recursively.
vim filename
Opens a file with the vim text editor. Using vim is a whole other blog post, but it is a powerful tool that many developers swear by.
sed
Used to perform text transformations on an input stream (a file or input from a pipeline). It can be used to search for strings in a file, update content, and replace content.
awk
Another big topic, awk
can run programs written in the AWK language that are used to transform text in a file. awk
can pattern match and perform operations on matched lines, useful for file formatting.
pipes
Piping is a process of sending the output of one command as the input to another. The syntax for a pipe is |
and information flows from left to right. So, to pipe the output of ls
into less
for example, you could write ls | less
, and enter the less
navigation with all the filenames in the directory - the output of ls
.
xargs
Converts standard input to arguments. Useful with commands that only accept arguments and do not read standard input such as echo
and rm
. With piping, input can be piped to xargs
which will execute a given command for each input. A common use of this is with the find
command. Results of find
can be piped to xargs
to run an operation on each result, such as rm
. For example, to remove any file that begin with test
, I could write find . -name 'test*' | xargs rm
.
And there's so much more
The internet is your friend when it comes to learning the details of Unix utilities! Don't be afraid to play around in your terminal either and see what you can do. Hopefully this is a good start for some useful commands to know about and the basics of how to use them. Thanks for reading!
Posted on August 7, 2020
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.