How do commands work?

In Unix/Linux systems, commands in a Unix shell are all programs. Anything you enter into the shell is called standard input. Similarly, anything printed on the shell is standard output. programs are binary files, which contains computer readable code. By default, all commands/programs invoked from the shell are located somewhere in the system. Not all programs in the system can be directly called in the shell. For example,

$ echo "echo is a program located in /bin"
echo is a program located in /bin/echo

echo is a common command in Linux/Unix. It is in fact a program located in the directory /bin. Let’s see where it is located.

$ whereis echo
echo: /bin/echo /usr/share/man/man1/echo.1.gz

How does the shell know where it is located? The most direct way of executing a program in the shell is to enter the absolute/relative path of the prgram followed by parameter. For example,

$ /bin/echo "This is a parameter"
This is a parameter

/bin/echo is the path echo, and the string is a parameter. The Environment variable $PATH tells the shell where all the commands are from. Internally, the shell looks for the program in the directories listed in $PATH ($var indicates variable in the Bash shell). If you want to make a program into a command and run it without entering its path, you will need to manually add the path into $PATH. For one-time use, run the command to add a path into $PATH. For example,

$ PATH=$PATH:~/opt/bin

Or, if you want to make the variable available to other programs,

$ export PATH=$PATH:~/opt/bin

Add the command into your bash profile such as ~/.bash_profile, so that it will be executed everytime a bash instance is open.

Some Useful Bash Commands

Manual

man is the most useful command ever. It prints the manual page of system tools. For example,

$ man du

This gives you the full manual of the program “disk usage”.

Redirect std I/O

Redirect standard output to a file. Overwrite the given file if it already exists.

$ program > filename # redirect output to a file
$ program >> filename # append

# is comments.

Redirect input from a file. This can be used to input complex parameters, or automated execution. This will give an error if the file doesn’t exist.

$ program < filename # redirect input from a file

Redirect stderr to a file. This will overwrite the given file if it already exists.

$ program 2> filename  # redirect error to a file
$ program 2>> filename #append

If you want clean output, redirect some outputs to the black hole.

$ program 2> /dev/null  # not printing error
$ program > /dev/null   # not printing anything

Pipe

Pipes | use the output of a program as the input of another program

$ program1 | program2 
$ ls | grep "file" # searching file
$ cat file.output | grep "text" # searching text

Standard Wildcard

Char Meaning
* any number of characters
? any single character
[ab] a or b
[a-c] a to c
{foo, bar} String foo or bar

Bash also support regular expression, which is powerful for pattern matching.

Remove

rm is a dangerous command. rmtrash is an alternative to rm that puts files using trush-put.

Run a program without getting out of the shell

$ program &

Use process control commands to manage them, such as ps, jobs, fg, bg, kill.

Measure the execution time of a program

Simplest way:

$ time program

Reference

Below tables are derived from here.

Category Command Description
basic shell clear clear all previous commands' output text from the terminal
exit (or logout) quits the shell
alias, unalias give a pseudonym to another command
(you may need to enclose the command in quotes if it contains spaces or operators)
history show a list of all past commands you have typed into this shell
directories ls list files in a directory
pwd displays the shell's current working directory
cd changes the shell's working directory to the given directory; can be a relative or absolute path
mkdir creates a new directory with the given name
rmdir removes the directory with the given name (the directory must be empty)
file operations cp copies a file/directory
mv moves (or renames) a file/directory
rm deletes a file
touch update the last-modified time of a file (or create an empty file)
file examination cat output the contents of a file
more (or less) output the contents of a file, one page at a time
head, tail output the beginning or ending of a file
wc output a count of the number of characters, lines, words, etc. in a file
du report disk space used by a file/directory. Personally I use du -sh * the most to check disk usage.
diff output differences between two files
file permissions chmod change the permissions on a file or group of files
chown change the owner of a file
chgrp change the group associated with a file
umask change the default permissions given to newly created files
searching and sorting grep search a file for a given string or expression
sort convert an input into a sorted output
uniq strip duplicate lines
find search for files by name within a given directory
xargs launch a command over each of a set of lines of input (often used with find)
locate search for files by name on the entire system
which shows the complete path of a command or file
compression zip, unzip create a .zip archive or extract its contents
tar Unix archiving/de-archiving program
gzip, gunzip GNU compression/decompression programs
bzip2, bunzip2 improved compression/decompression programs
system information date outputs the current date/time
cal outputs an ASCII calendar
uname print information about the system
time measure how long a program takes to run
process management ps, jobs list the processes you are running; every process has a unique integer id number (PID)
top see what processes are using the most CPU/memory, and show system memory/CPU stats
kill terminate a process
killall terminate a group of processes by name
^C or ^\ (hotkey) terminates (kills) the currently running process
^Z (hotkey) suspends the currently running process
& (special character) when & is placed at the end of a command, that command is run in the background (shell does not wait for the command to finish before returning to the input prompt)
bg, fg starts a suspended process running in the background or foreground
users and groups whoami outputs your user name
passwd changes your password
groups list the groups to which a user belongs
sudo execute a single command as the super-user
su log in to a shell as the super-user
multi-user environments hostname outputs the name of the current computer/server
w, finger see who is logged in to this computer
write sends a message to another user logged in to this computer
wall broadcasts a message to all other users logged in to this computer
.plan (filename) a special hidden file you can create in your home directory, whose contents will be displayed when other users run finger on you. Was originally intended to be used to tell others what you are up to right now. (the Twitter of the 1970s!)
network links, lynx text-only web browsers (yes, really)
ssh, sftp, scp connect to a remote Unix server; open a shell on it or send/receive files from it
wget download from a URL and save it to a file on the local hard drive
curl download from a URL and output its contents to the console
pine, mail text-only email programs
text editors pico, nano crappy but simple text editors (recommended)
emacs a complicated text editor (not recommended)
vi, vim another complicated text editor (not recommended)
regular expressions sed stream editor; find/replace based on regular expressions
egrep extended version of grep that matches regular expressions
programming javac, java compile or run a Java program
python, perl, ruby,
gcc, sml, ...		 compile or run programs in various other languages
shell scripting echo, printf like println for the shell; outputs a message or value
read reads a value from standard input
set, unset give values to a variable, or delete a variable
export sets a variable that any sub-programs launched by this shell can see
let for computing integer variable values
source executes commands/statements stored in another file
(useful for re-loading .bash_profile without logging out)
if, [, for, while bash control statements
seq outputs a sequence of integers (used with for loops)
miscellaneous yes output "y" (or another string) over and over
sleep, usleep pause for a given number of seconds or ms
~stepp/cowsay displays a talking ASCII cow (on attu only, though you could install it if you are using Linux on a PC)
xeyes googley eyes that follow your mouse cursor
build management make determine which parts of a system must be recompiled, and compile them
svn, cvs Subversion and CVS version-control systems

Keyboard Shortcuts

key / character description
^C or ^\ kills the currently running process
^D end-of-input; press this if a program is reading input from your keyboard and you want to notify it that you are finished
^Z suspends (pauses) the currently running process; use fg or bg to resume it
^S never ever press this; worst hotkey ever; totally locks up your shell until you press ^Q